Another scripting language implemented into Python itself?

Q

Quest Master

I am interested in developing an application where the user has an
ample amount of power to customize the application to their needs, and
I feel this would best be accomplished if a scripting language was
available. However, I want to code this application in Python, and I
have not yet heard of an implementation of another scripting language
into Python.

An example of what I mean is this (an implementation of Lua into Ruby
-- which I'd prefer not to use): http://ruby-lua.unolotiene.com/

I know C/C++ might be better suited for a task of this kind, but most
of the modules in my application which need speed have already been
coded in C++. I want to use Python as the "glue" for this project;
writing an entire application with a pretty GUI is not fun, as most of
you may know from previous experience.

So, my question is simply this: is there an implementation of another
scripting language into Python?
 
R

Roy Smith

Quest Master said:
I am interested in developing an application where the user has an
ample amount of power to customize the application to their needs, and
I feel this would best be accomplished if a scripting language was
available. However, I want to code this application in Python, and I
have not yet heard of an implementation of another scripting language
into Python.

An example of what I mean is this (an implementation of Lua into Ruby
-- which I'd prefer not to use): http://ruby-lua.unolotiene.com/

I know C/C++ might be better suited for a task of this kind, but most
of the modules in my application which need speed have already been
coded in C++. I want to use Python as the "glue" for this project;
writing an entire application with a pretty GUI is not fun, as most of
you may know from previous experience.

So, my question is simply this: is there an implementation of another
scripting language into Python?

Python *is* a scripting language. Why not just let your users write
Python modules which you them import and execute via some defined API?
 
R

Rocco Moretti

Roy said:
Python *is* a scripting language. Why not just let your users write
Python modules which you them import and execute via some defined API?

I can think of a couple of reasons off the top of my head:

The OP doesn't mention his application, but there is something to be
said about domain specific scripting languages. A well designed
domain-specific scripting language(*) with the appropriate high level
constructs can make script writing simpler.

There's also an audience factor. Python is easy to learn, but it's still
a programming language. A well designed domain-specific scripting
language(*) can make it very easy to get work done in a particular case
without having to learn the Python mind frame. (How assignments work,
mutable/immutable, Python's call passing system, etc.)

Python's also dangerous. Every time you do an "import module", you put
your system at risk of crashing, having the hard-drive wiped, sending
incorrect signal codes to peripherals, etc. A well-designed specialty
language(*) minimizes those risks - don't need disk access? Don't allow
it in your language.

There's also the valuable learning experience of designing and
implementing a scripting language.

(*) Note that I keep saying "well-designed". A poorly designed scripting
language is very bad - you can feel shackled by the restrictions it
imposes, and find yourself unable to do what you want without laborious
contortions, if at all. I also say "domain specific" because, at this
time, there are enough general purpose scripting languages that you are
likely to find what you need already existing, unless you are
experimeting with a completely new programing paradigm.

To answer the OP's question:

Yes - sort of. Currently work is underway to implement the Python
scripting language in Python. Cleverly called "PyPy", the website is
"http://www.codespeak.net/pypy".
 
T

Terry Reedy

Roy Smith said:
Python *is* a scripting language. Why not just let your users write
Python modules which you them import and execute via some defined API?

That was my thought also, with the note that you only need to document as
much users will need, including specific modules (if any) to import.

Terry J. Reedy
 
J

Jeff Shannon

Roy said:
Python *is* a scripting language. Why not just let your users write
Python modules which you them import and execute via some defined API?

Because you cannot make Python secure against a malicious (or
ignorant) user -- there's too much flexibility to be able to guard
against every possible way in which user-code could harm the system.
Parsing your own (limited) scripting language allows much better
control over what user-code is capable of doing, and therefore allows
(at least some measure of) security against malicious code.

To the O.P.: Yes, people have implemented other languages in Python.
For example, I believe that Danny Yoo has written a Scheme
interpreter in Python (Google tells me it should be at
http://hkn.eecs.berkeley.edu/~dyoo/python/pyscheme but I'm getting no
response from that host right now), but I don't know whether Scheme
counts as a scripting language. ;)

However, if you're using a fully-featured language for these user
scripts, you'll probably have the same security issues I mentioned for
Python. Unless you really need that level of features, you may be
better off designing your own limited language. Check into the docs
for pyparsing for a starter...

Jeff Shannon
Technician/Programmer
Credit International
 
R

Roy Smith

Rocco Moretti said:
The OP doesn't mention his application, but there is something to be
said about domain specific scripting languages. A well designed
domain-specific scripting language(*) with the appropriate high level
constructs can make script writing simpler.

This is a bit of a sore point with me.

I've been involved with several projects where people felt the need to
invent their own scripting languages. It usually starts with "we don't
need the power of a full programming language, we only need to be able
to do X, Y, and Z". So they write a little language which lets them do
X, Y, and Z.

Then they discover they need more complex data structures than they
originally thought. And nested loops. And functions. And more
sophisticated variable scoping rules. And a regex library. And 47
other things. So they duct-tape all those into the system.

A few years later, you end up with most of a real programming language,
except with a whole bunch of warts.

The syntax is usually quirky (the one I'm working with today does not
allow any space before the open paren of a function call, but requires
it before the opening paren of an "if" statement). It generally has
poor error reporting. It doesn't have the whole family of free tools
that grow up around any real language (editor customization packages,
syntax checkers, debuggers, extensions, etc). You doesn't have a gaggle
of tutorial books written about it that you can buy from your favorite
on-line bookseller.

Worse, when you need more brains/bodies on the project, you can't put an
add on Hot Jobs for "experienced OurOwnScriptingLanguage programmer" and
expect to get anybody who can be productive quickly.

What it does have is a body of code dependent on it which is now so
large that porting it to something else is an unthinkably large task.
And it's got a small cadre of language gurus who spend all day defending
the language with answers like, "But, it was never *intended* that
people would do stuff like this with it".

Anyway, that's my little rant on inventing your own scripting language.
Imbed Python, or Perl, or TCL, or Ruby, or PHP, or Java, or whatever
floats your boat. Almost any choice has got to be better than rolling
your own. Invest your intellectual capital doing what you can do best,
and don't get bogged down developing a new language.
 
J

Jack Diederich

This is a bit of a sore point with me.

I've been involved with several projects where people felt the need to
invent their own scripting languages. It usually starts with "we don't
need the power of a full programming language, we only need to be able
to do X, Y, and Z". So they write a little language which lets them do
X, Y, and Z.

Then they discover they need more complex data structures than they
originally thought. And nested loops. And functions. And more
sophisticated variable scoping rules. And a regex library. And 47
other things. So they duct-tape all those into the system.

A few years later, you end up with most of a real programming language,
except with a whole bunch of warts.

The syntax is usually quirky (the one I'm working with today does not
allow any space before the open paren of a function call, but requires
it before the opening paren of an "if" statement). It generally has
poor error reporting. It doesn't have the whole family of free tools
that grow up around any real language (editor customization packages,
syntax checkers, debuggers, extensions, etc). You doesn't have a gaggle
of tutorial books written about it that you can buy from your favorite
on-line bookseller.

Worse, when you need more brains/bodies on the project, you can't put an
add on Hot Jobs for "experienced OurOwnScriptingLanguage programmer" and
expect to get anybody who can be productive quickly.

What it does have is a body of code dependent on it which is now so
large that porting it to something else is an unthinkably large task.
And it's got a small cadre of language gurus who spend all day defending
the language with answers like, "But, it was never *intended* that
people would do stuff like this with it".

Me Too!
I mean, did you used to work at CDNOW too?
I don't miss that want-to-gouge-out-your-own-eyes feeling.

-Jack
 
B

Bob Smith

Rocco said:
Python's also dangerous. Every time you do an "import module", you put
your system at risk of crashing, having the hard-drive wiped

Have you been drinking again?
 
C

Carl Banks

Roy said:
This is a bit of a sore point with me.

I've been involved with several projects where people felt the need to
invent their own scripting languages. It usually starts with "we don't
need the power of a full programming language, we only need to be able
to do X, Y, and Z". So they write a little language which lets them do
X, Y, and Z.

Then they discover they need more complex data structures than they
originally thought. And nested loops. And functions. And more
sophisticated variable scoping rules. And a regex library. And 47
other things. So they duct-tape all those into the system.

Not only is it short-sighted, I would say it is quite arrogant to
believe you can anticipate every possible use of this script you're
going to impliement, and can safely hold back essential language
features.

Anyway, that's my little rant on inventing your own scripting language.
Imbed

EMBED.

This spelling error wouldn't bother me if I didn't work with people
whose own job title is embedded control engineer, yet who still
misspell it "imbedded."

Python, or Perl, or TCL, or Ruby, or PHP,

Not PHP. PHP is one of the better (meaning less terrible) examples of
what happens when you do this sort of thing, which is not saying a lot.
PHP was originally not much more than a template engine with some
crude operations and decision-making ability. Only its restricted
problem domain has saved it from the junkheap where it belongs.

TCL isn't that great in this regard, either, as it makes a lot of
common operations that ought to be very simple terribly unweildy.

or Java, or whatever
floats your boat. Almost any choice has got to be better than rolling
your own. Invest your intellectual capital doing what you can do best,
and don't get bogged down developing a new language.

You're right. I use a custom, domain-specific language in my work.
Whenever I use it, all I can think of is how much better this POS would
be if they had just extended Python (probably would even be faster).
At least they were smart enough to (try to) make it into a complete
programming language.
 
B

Bengt Richter

On 24 Jan 2005 19:46:43 -0800 said:
EMBED.

This spelling error wouldn't bother me if I didn't work with people
whose own job title is embedded control engineer, yet who still
misspell it "imbedded."
wordnet seems to accept it as an alternative spelling (not too recent):

[20:44] C:\pywk\sovm>wn imbed -over

Overview of verb imbed

The verb imbed has 1 sense (first 1 from tagged texts)

1. implant, engraft, embed, imbed, plant -- (to fix or set securely or deeply: "Kneeling, Cobb p
lanted a sturdy knee in the small of his back,")

funny that googling define:imbed gets nothing. Nor define:embed ??
I've had a number of misses in that kind of search. I wonder if it's
temporarily broken (or maybe it hit a licensing snag with the various
online dictionaries?)

Regards,
Bengt Richter
 
O

Orlando Vazquez

Jeff Shannon wrote:

snip
Because you cannot make Python secure against a malicious (or ignorant)
user -- there's too much flexibility to be able to guard against every
possible way in which user-code could harm the system. Parsing your own
(limited) scripting language allows much better control over what
user-code is capable of doing, and therefore allows (at least some
measure of) security against malicious code.

I don't see how that would equate to something that the original
programmer should be concerned about. You could include a bit in your
licensing scheme that voids all support on code that has been modified
in any way. You shouldn't be obligated and no one expects you to support
something the end-user has mucked with.

You could trivially enforce this by keeping checksums of all the system
files.

In any case, there's nothing you can really do to "secure" your code.
This is true of any language, C, C++, and especially scripting languages
like Python. Anyone who has the determination get at and modify the code
probably will.

The only time where I can see someone using another language in place of
Python for a scripting language is just domain-specific factors, e.g. if
you need the extension language to be easily used non-programmers.

Just a thought.
 
O

Ola Natvig

Roy said:
Python *is* a scripting language. Why not just let your users write
Python modules which you them import and execute via some defined API?

There are examples of XML used for application configuration and in
template systems, I don't know how much power you intend to give your
users, but if it boils down to simple display logic and some small
'macro' calls to a predefined API, you could make some kind of tagged
language for that implementing basic flowcontroll functionality,
allowing function calling and definition.

You could look at wxWidgets/wxPython's XRC format, which are a XML
tagged format that are used to customize layout of wx forms.

But I don't think there are a implemented any fully usable scripting
languages like that in Python. But it should not be to much work if you
don't need many functions.
 
A

Arthur

Python *is* a scripting language. Why not just let your users write
Python modules which you them import and execute via some defined API?

This is the tact I take with PyGeo.

The burden is on myself, as the API designer and documenter to make it
as accessible as I might want it to be. And I think I can accomplish
that without building in crippling limitations as to what depth a
"scripter" might want to explore - even to the extent of extending the
API itself.

I guess its a matter of the audience one hopes to reach. There are a
number of other dynamic geometry applications out there. Despite my
own interest in the subject, it was precisely the limitations their
API, whether GUI based or otherwise, that led me to the explorations
that is becoming PyGeo. It was the limitaions in exploring *geometry*
to which I refer. Having no per se interest in programming, I would
not impose the assumption of such an interest on my users.

I guess I hope to reach others like myself. With no particular
interest in programming, per se. Just with what you can get do with
it.

Art
 
R

Roy Smith

Carl Banks said:

My apologies for being sloppy. And with an initial capital, so it just
jumps off the page at you :)
Not PHP. PHP is one of the better (meaning less terrible) examples of
what happens when you do this sort of thing, which is not saying a lot.

But, that's exactly my point. To be honest, I've never used PHP. But
however bad it may be, at least it's got a few years of people fixing
bugs, writing books, writing add-on tools, etc, behind it. Better to
use somebody else's well-known and well-supported mess of a scripting
language than to invest several person-years inventing your own mess
that's no better.

There are a lot of existing scripting languages to pick from. It's nice
to pick the best one, but even if you pick the worst, that's probably
better than you can do on your own.
TCL isn't that great in this regard, either, as it makes a lot of
common operations that ought to be very simple terribly unweildy.

In my last job, I did a lot of TCL. I've posted on this before (when I
was at a previous employer), so I'll just provide a pointer
(http://tinyurl.com/44w6n). That article says most of what that needs
saying, *AND* proves that I really do know how to spell embed :)

It might be worth adding, however, that the TCL implementation discussed
above was a 3rd generation for that product. Generation #1 was a bunch
of shell scripts. Generation #2 was a home-grown scripting language.
Fortunately, they learned their lesson soon enough to rescue the project
with a conversion to TCL. I'm not sure how many person-years were
wasted both in the initial implementation and in the conversion effort,
but I imagine it was a $500,000 mistake.
 
N

Nick Coghlan

Fuzzyman said:
An implementation of the core language semantics - without any modules
or file operations would be dead useful.

It could replace some of the function of the long dead rexec modules as
well as support projects like this.

Securing a custom build of the CPython interpreter would probably be
significantly easier than designing a 'secure mode' that ran on top of the
standard version. The former might even be a stepping stone towards the latter.

Still not easy though (the main task would be to prevent Python code from
accessing the OS, while still allowing module imports to work).

Cheers,
Nick.
 
F

Fuzzyman

Nick said:
Securing a custom build of the CPython interpreter would probably be
significantly easier than designing a 'secure mode' that ran on top of the
standard version. The former might even be a stepping stone towards the latter.

Still not easy though (the main task would be to prevent Python code from
accessing the OS, while still allowing module imports to work).

Pure python imports only... no C extensions.

Regards,

Fuzzy
http://www.voidspace.org.uk/python/index.shtml
 
C

Cameron Laird

Jeff Shannon wrote:

snip


I don't see how that would equate to something that the original
programmer should be concerned about. You could include a bit in your
licensing scheme that voids all support on code that has been modified
in any way. You shouldn't be obligated and no one expects you to support
something the end-user has mucked with.

You could trivially enforce this by keeping checksums of all the system
files.

In any case, there's nothing you can really do to "secure" your code.
This is true of any language, C, C++, and especially scripting languages
like Python. Anyone who has the determination get at and modify the code
probably will.

The only time where I can see someone using another language in place of
Python for a scripting language is just domain-specific factors, e.g. if
you need the extension language to be easily used non-programmers.
.
.
.
I think there's a bit of "talking past" each other.
There's a serious issue here that I suspect Mr.
Vazquez misunderstood. I'll try to illustrate:

The original poster wants to work in Python. That's
fine. Several of us have suggested he further
expose Python itself to his end-users as an extension
language. That certainly is feasible. He needn't
explain all of Python to those end-users--probably
only a bit about "assignments", control structures,
and maybe lists.

That approach creates a sort of fragility, though.
Python includes, along with much else, os.unlink().
Suppose our original poster doesn't want end-users
to be able to delete files (or directories ...).
That particular design decision is NOT particularly
apt for a licensing specification, much as I generally
favor trust in the latter; don't-delete-filesystem-
entries is simply too low-level to admit good
expression in legal language. More broadly, the
model of "end-users mucking around" captures the
range of concerns only poorly.

This is a serious issue.

It's also one that brings Tcl, mentioned several
times in this thread, back into focus. Tcl presents
the notion of "safe interpreter", that is, a sub-
ordinate virtual machine which can interpret only
specific commands. It's a thrillingly powerful and
correct solution to the main problem Jeff and others
have described.
 
R

Roy Smith

Cameron Laird said:
It's also one that brings Tcl, mentioned several
times in this thread, back into focus. Tcl presents
the notion of "safe interpreter", that is, a sub-
ordinate virtual machine which can interpret only
specific commands. It's a thrillingly powerful and
correct solution to the main problem Jeff and others
have described.

Yup, we used that feature. I don't remember the details, but I do
remember that you couldn't open files, and the source command only
allowed access to files in a specific directory.
 
C

Cameron Laird

.
.
.
Not PHP. PHP is one of the better (meaning less terrible) examples of
what happens when you do this sort of thing, which is not saying a lot.
PHP was originally not much more than a template engine with some
crude operations and decision-making ability. Only its restricted
problem domain has saved it from the junkheap where it belongs.

TCL isn't that great in this regard, either, as it makes a lot of
common operations that ought to be very simple terribly unweildy.
.
.
.
I've lost track of the antecedent by the time of our arrival at
"this regard". I want to make it clear that, while Tcl certainly
is different from C and its imitators, and, in particular, insists
that arithmetic be expressed more verbosely than in most languages,
the cause is quite distinct from the imperfections perceived in
PHP. PHP is certainly an instance of "scope creep" in its semantics.
Tcl was designed from the beginning, though, and has budged little in
over a decade in its fundamentals; Tcl simply doesn't bother to "make
a lot of common operations ..." concise.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,482
Members
44,901
Latest member
Noble71S45

Latest Threads

Top