looking for tips on how to implement "ruby-style" Domain SpecificLanguage in Python

Discussion in 'Python' started by mark, Jan 6, 2009.

  1. mark

    mark Guest

    I want to implement a internal DSL in Python. I would like the syntax
    as human readable as possible. This means no disturbing '.;()\'
    characters. I like to have the power of the hosting language as well.
    Thats why I want to build it as an internal DSL and NOT as a external
    DSL.

    I want the DSL as human readable as possible:
    >>>

    open_browser

    navigate_to 'www.openstreetmap.org' website

    search 'Von-Gumppenberg-Strasse, Schmiechen'

    verify search_result

    zoom in
    <<<

    Martin Fowler recommends "Method Chaining" to build internal DSLs:
    >>>

    Browser("http://www.openstreetmap.org/") \
    .search("Von-Gumppenberg-Strasse, Schmiechen") \
    .zoom_in()
    <<<

    I guess that it is possible to argue that this means the same.
    Nevertheless I do not like all the parentheses and punctuation
    necessary to satisfy the Python interpreter.

    The reason why I need this is that I want to have non technical people
    review the files written in the DSL.

    I already know that there are parser frameworks available but I want
    to build it as internal DSL in Python (Yes, I know ANTLR, Ply, and
    whatnot).

    How would one approach this in Python? Do I need to build a custom
    loader which compiles *.dsl files to *.pyc files? Is it possible to
    switch between the custom DSL and the standard Python interpreter?
     
    mark, Jan 6, 2009
    #1
    1. Advertising

  2. mark

    sturlamolden Guest

    On Jan 6, 4:32 pm, mark <> wrote:
    > Is it possible to
    > switch between the custom DSL and the standard Python interpreter?


    As far as I can tell, there are three different options:

    - Embed a Python and DSL interpreter in the same executable.

    - Write the DSL interpreter in Python.

    - Expose the DSL interpreter as a Python extension module.


    I don't know which you prefer, but I would try to avoid the first.
     
    sturlamolden, Jan 6, 2009
    #2
    1. Advertising

  3. mark

    sturlamolden Guest

    On Jan 6, 4:32 pm, mark <> wrote:
    > I want to implement a internal DSL in Python. I would like the syntax
    > as human readable as possible.


    Also beware that Python is not Lisp. You cannot define new syntax (yes
    I've seen the goto joke).
     
    sturlamolden, Jan 6, 2009
    #3
  4. On Jan 6, 8:13 am, sturlamolden <> wrote:
    > On Jan 6, 4:32 pm, mark <> wrote:
    > >
    > > Is it possible to
    > > switch between the custom DSL and the standard Python interpreter?
    > >

    >
    > - Write the DSL interpreter in Python.
    >


    There are Python modules out there that make writing a language
    interpreter almost trivial, provided you are familiar with tools like
    Bison and the theories about parsing in general. I suggest PLY, but
    there are other really good solution out there.

    If you are familiar enough with parsing and the syntax is simple
    enough, you can write your own parser. The syntax you describe above
    is really simple, so using str.split and then calling a function based
    on the first item is probably enough.
     
    Jonathan Gardner, Jan 6, 2009
    #4
  5. On Jan 6, 8:18 am, sturlamolden <> wrote:
    > On Jan 6, 4:32 pm, mark <> wrote:
    >
    > > I want to implement a internal DSL in Python. I would like the syntax
    > > as human readable as possible.

    >
    > Also beware that Python is not Lisp. You cannot define new syntax (yes
    > I've seen the goto joke).


    This isn't really true. You can, for instance, write a program (in
    Python) that takes your pseudo-Python and converts it into Python.
    This is what a number of templating libraries such as Mako do.
     
    Jonathan Gardner, Jan 6, 2009
    #5
  6. mark

    Kay Schluehr Guest

    > How would one approach this in Python? Do I need to build a custom
    > loader which compiles *.dsl files to *.pyc files? Is it possible to
    > switch between the custom DSL and the standard Python interpreter?


    Sure, but there is no way to avoid extending the Python parser and
    then your DSL becomes external.

    I remember having had a similar discussion a while ago with Kevin
    Dangoor the original TurboGears developer who has also written Paver
    [1]. In the end DSL syntax wasn't worth the hassle and Kevin developed
    Paver entirely in Python.

    Kay

    [1] http://www.blueskyonmars.com/projects/paver/
     
    Kay Schluehr, Jan 6, 2009
    #6
  7. On 2009-01-06 20:42, Kay Schluehr wrote:
    >> How would one approach this in Python? Do I need to build a custom
    >> loader which compiles *.dsl files to *.pyc files? Is it possible to
    >> switch between the custom DSL and the standard Python interpreter?


    > Sure, but there is no way to avoid extending the Python parser and
    > then your DSL becomes external.


    Try python4ply:

    http://dalkescientific.com/Python/python4ply.html

    ....much easier to work with than extending the Python parser by hand.

    --
    Marc-Andre Lemburg
    eGenix.com

    Professional Python Services directly from the Source (#1, Jan 06 2009)
    >>> Python/Zope Consulting and Support ... http://www.egenix.com/
    >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
    >>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/

    ________________________________________________________________________

    ::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


    eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
    Registered at Amtsgericht Duesseldorf: HRB 46611
    http://www.egenix.com/company/contact/
     
    M.-A. Lemburg, Jan 6, 2009
    #7
  8. mark

    Steve Holden Guest

    J Kenneth King wrote:
    [...]
    > I could go on a really long rant about how the two are worlds apart, but
    > I'll let Google tell you if you're really interested.


    a) How is Google going to know if he's really interested?

    b) Put a space after the "--" in your sig, please; that way my mailer
    won't yto to quote your signature as part of the message.

    regards
    Steve
    --
    Steve Holden +1 571 484 6266 +1 800 494 3119
    Holden Web LLC http://www.holdenweb.com/
     
    Steve Holden, Jan 6, 2009
    #8
  9. On Jan 6, 12:24 pm, J Kenneth King <> wrote:
    > Jonathan Gardner <> writes:
    > > On Jan 6, 8:18 am, sturlamolden <> wrote:
    > >> On Jan 6, 4:32 pm, mark <> wrote:

    >
    > >> > I want to implement a internal DSL in Python. I would like the syntax
    > >> > as human readable as possible.

    >
    > >> Also beware that Python is not Lisp. You cannot define new syntax (yes
    > >> I've seen the goto joke).

    >
    > > This isn't really true. You can, for instance, write a program (in
    > > Python) that takes your pseudo-Python and converts it into Python.
    > > This is what a number of templating libraries such as Mako do.

    >
    > Which is not even close to being the same.
    >
    > Lisp - the program source is also the data format
    >
    > Python - the program source is a string
    >
    > I could go on a really long rant about how the two are worlds apart, but
    > I'll let Google tell you if you're really interested.


    I get that Lisp is special because you can hack on the reader as it is
    reading the file in. This is strongly discouraged behavior, as far as
    I know, despite the number of cute hacks you can accomplish with it.

    But consider that this really isn't different than having a program
    read in the lisp-with-modification source and spitting out pure lisp,
    to be read by an honest-to-gosh lisp program later.

    If that's the case, then Lisp and Python really aren't that different
    in this regard, except that you don't have the option of modifying the
    reader as it reads in the file.
     
    Jonathan Gardner, Jan 6, 2009
    #9
  10. Re: looking for tips on how to implement "ruby-style" Domain Specific Language in Python

    Jonathan Gardner <> writes:

    > On Jan 6, 12:24 pm, J Kenneth King <> wrote:
    >> Jonathan Gardner <> writes:
    >> > On Jan 6, 8:18 am, sturlamolden <> wrote:
    >> >> On Jan 6, 4:32 pm, mark <> wrote:

    >>
    >> >> > I want to implement a internal DSL in Python. I would like the syntax
    >> >> > as human readable as possible.

    >>
    >> >> Also beware that Python is not Lisp. You cannot define new syntax (yes
    >> >> I've seen the goto joke).

    >>
    >> > This isn't really true. You can, for instance, write a program (in
    >> > Python) that takes your pseudo-Python and converts it into Python.
    >> > This is what a number of templating libraries such as Mako do.

    >>
    >> Which is not even close to being the same.
    >>
    >> Lisp - the program source is also the data format
    >>
    >> Python - the program source is a string
    >>
    >> I could go on a really long rant about how the two are worlds apart, but
    >> I'll let Google tell you if you're really interested.

    >
    > I get that Lisp is special because you can hack on the reader as it is
    > reading the file in. This is strongly discouraged behavior, as far as
    > I know, despite the number of cute hacks you can accomplish with it.


    It is generally discouraged unless there's a reason for it.

    > But consider that this really isn't different than having a program
    > read in the lisp-with-modification source and spitting out pure lisp,
    > to be read by an honest-to-gosh lisp program later.
    >
    > If that's the case, then Lisp and Python really aren't that different
    > in this regard, except that you don't have the option of modifying the
    > reader as it reads in the file.


    I think you are missing the distinction.

    Lisp expressions are also data structures. A Lisp expression can be
    passed to functions and macros to be operated on before being
    executed. When you're writing Lisp source, you're basically looking at
    the AST on one level and when you start writing macros for your program,
    you're creating a "DSL" or interface to that AST. Lisp source is
    eventually expanded to a giant list that is consed by the evaluator (as
    far as I understand it. I'm just getting into the compiler stuff
    myself).

    Consider:

    (my-func 1 2 3)

    This is just a list, the "primitive" data-type in Lisp! This piece of
    "data" can be operated on by other bits of Lisp code because it is just
    a list as far as Lisp is concerned.

    In contrast, Python source is a string that needs to be parsed into
    bytecode which is then run through the interpreter. The AST is
    completely hidden from the source author. Python expressions are not
    data types either and hence no macros -- I can't write a python function
    that generates python code at compile time. I can only write a python
    program that parses some other string and generates code that can be run
    by another interpreter.

    Consider:

    for i in range(0, 100):
    do_something_interesting(i)

    That's a pretty straight forward Python expression, but I can't do
    anything with it -- it's not a unit of data, it's a string.

    The distinction is not subtle by any means.
     
    J Kenneth King, Jan 7, 2009
    #10
  11. mark

    Kay Schluehr Guest

    On 7 Jan., 16:50, J Kenneth King <> wrote:

    > Python expressions are not
    > data types either and hence no macros -- I can't write a python function
    > that generates python code at compile time.


    Have you ever considered there are languages providing macros other
    than Lisp? Macros have nothing to do with homoiconcity.

    > I can only write a python
    > program that parses some other string and generates code that can be run
    > by another interpreter.


    No, it is the same interpreter and it is also possible to modify
    python parsers on the fly. This is just not possible with Pythons
    builtin parser.


    >
    > Consider:
    >
    > for i in range(0, 100):
    >     do_something_interesting(i)
    >
    > That's a pretty straight forward Python expression, but I can't do
    > anything with it -- it's not a unit of data, it's a string.
    >
    > The distinction is not subtle by any means.
     
    Kay Schluehr, Jan 7, 2009
    #11
  12. mark

    Chris Mellon Guest

    On Wed, Jan 7, 2009 at 9:50 AM, J Kenneth King <> wrote:
    > Jonathan Gardner <> writes:
    >
    >> On Jan 6, 12:24 pm, J Kenneth King <> wrote:
    >>> Jonathan Gardner <> writes:
    >>> > On Jan 6, 8:18 am, sturlamolden <> wrote:
    >>> >> On Jan 6, 4:32 pm, mark <> wrote:
    >>>
    >>> >> > I want to implement a internal DSL in Python. I would like the syntax
    >>> >> > as human readable as possible.
    >>>
    >>> >> Also beware that Python is not Lisp. You cannot define new syntax (yes
    >>> >> I've seen the goto joke).
    >>>
    >>> > This isn't really true. You can, for instance, write a program (in
    >>> > Python) that takes your pseudo-Python and converts it into Python.
    >>> > This is what a number of templating libraries such as Mako do.
    >>>
    >>> Which is not even close to being the same.
    >>>
    >>> Lisp - the program source is also the data format
    >>>
    >>> Python - the program source is a string
    >>>
    >>> I could go on a really long rant about how the two are worlds apart, but
    >>> I'll let Google tell you if you're really interested.

    >>
    >> I get that Lisp is special because you can hack on the reader as it is
    >> reading the file in. This is strongly discouraged behavior, as far as
    >> I know, despite the number of cute hacks you can accomplish with it.

    >
    > It is generally discouraged unless there's a reason for it.
    >
    >> But consider that this really isn't different than having a program
    >> read in the lisp-with-modification source and spitting out pure lisp,
    >> to be read by an honest-to-gosh lisp program later.
    >>
    >> If that's the case, then Lisp and Python really aren't that different
    >> in this regard, except that you don't have the option of modifying the
    >> reader as it reads in the file.

    >
    > I think you are missing the distinction.
    >
    > Lisp expressions are also data structures. A Lisp expression can be
    > passed to functions and macros to be operated on before being
    > executed. When you're writing Lisp source, you're basically looking at
    > the AST on one level and when you start writing macros for your program,
    > you're creating a "DSL" or interface to that AST. Lisp source is
    > eventually expanded to a giant list that is consed by the evaluator (as
    > far as I understand it. I'm just getting into the compiler stuff
    > myself).
    >
    > Consider:
    >
    > (my-func 1 2 3)
    >
    > This is just a list, the "primitive" data-type in Lisp! This piece of
    > "data" can be operated on by other bits of Lisp code because it is just
    > a list as far as Lisp is concerned.
    >
    > In contrast, Python source is a string that needs to be parsed into
    > bytecode which is then run through the interpreter. The AST is
    > completely hidden from the source author. Python expressions are not
    > data types either and hence no macros -- I can't write a python function
    > that generates python code at compile time. I can only write a python
    > program that parses some other string and generates code that can be run
    > by another interpreter.
    >
    > Consider:
    >
    > for i in range(0, 100):
    > do_something_interesting(i)
    >
    > That's a pretty straight forward Python expression, but I can't do
    > anything with it -- it's not a unit of data, it's a string.
    >
    > The distinction is not subtle by any means.



    Ignoring reader macros for a moment, there is no way in either lisp,
    ruby, or python to change the syntax that the compiler understands,
    and the ability to work with your code directly as a data structure
    (which is what makes lisp macros powerful) isn't directly relevant to
    the idea of an "internal' DSL.

    The OP wants a Ruby-style DSL by which he means "something that lets
    me write words instead of expressions". The ruby syntax is amenable to
    this, python (and lisp, for that matter) syntax is not and you can't
    implement that style of internal DSL in those languages.

    The answer to the OP is "you can't - use Ruby or modify your requirements".
     
    Chris Mellon, Jan 7, 2009
    #12
  13. Re: looking for tips on how to implement "ruby-style" Domain Specific Language in Python

    Kay Schluehr <> writes:

    > On 7 Jan., 16:50, J Kenneth King <> wrote:
    >
    >> Python expressions are not
    >> data types either and hence no macros -- I can't write a python function
    >> that generates python code at compile time.

    >
    > Have you ever considered there are languages providing macros other
    > than Lisp?


    Of course.

    > Macros have nothing to do with homoiconcity.


    Not directly, no.

    >> I can only write a python
    >> program that parses some other string and generates code that can be run
    >> by another interpreter.

    >
    > No, it is the same interpreter and it is also possible to modify
    > python parsers on the fly. This is just not possible with Pythons
    > builtin parser.


    PyPy is probably the best bet when/if it gets finished.

    >
    >
    >>
    >> Consider:
    >>
    >> for i in range(0, 100):
    >>     do_something_interesting(i)
    >>
    >> That's a pretty straight forward Python expression, but I can't do
    >> anything with it -- it's not a unit of data, it's a string.
    >>
    >> The distinction is not subtle by any means.
     
    J Kenneth King, Jan 7, 2009
    #13
  14. mark

    Carl Banks Guest

    On Jan 6, 9:32 am, mark <> wrote:
    > I want to implement a internal DSL in Python. I would like the syntax
    > as human readable as possible. This means no disturbing '.;()\'
    > characters. I like to have the power of the hosting language as well.
    > Thats why I want to build it as an internal DSL and NOT as a external
    > DSL.
    >
    > I want the DSL as human readable as possible:
    >
    > open_browser
    >
    > navigate_to 'www.openstreetmap.org'website
    >
    > search 'Von-Gumppenberg-Strasse, Schmiechen'
    >
    > verify search_result
    >
    > zoom in


    In the Python grammar, there are no non-trivial situations where two
    expressions can be separated by whitespace and not punctuation. (The
    trivial exception is string concatentation.)

    String constants like 'www.openstreetmap.org' and identifiers like
    open_browser are expressions, and if you try to separate them with
    whitespace you get a syntax error.

    So you can't make an internal DSL like this that uses Python's built-
    in grammar. You'd have to hack the parser or settle for an external
    preprocessor.



    > Martin Fowler recommends "Method Chaining" to build internal DSLs:
    >
    >  Browser("http://www.openstreetmap.org/") \
    >         .search("Von-Gumppenberg-Strasse, Schmiechen") \
    >         .zoom_in()
    >  <<<
    >
    > I guess that it is possible to argue that this means the same.
    > Nevertheless I do not like all the parentheses and punctuation
    > necessary to satisfy the Python interpreter.
    >
    > The reason why I need this is that I want to have non technical people
    > review the files written in the DSL.
    >
    > I already know that there are parser frameworks available but I want
    > to build it as internal DSL in Python (Yes, I know ANTLR, Ply, and
    > whatnot).
    >
    > How would one approach this in Python? Do I need to build a custom
    > loader which compiles *.dsl files to *.pyc files? Is it possible to
    > switch between the custom DSL and the standard Python interpreter?


    I don't know specifically what you mean my "custom loader", or
    "switching between the custom DSL and the standard Python
    interpreter".

    However, the gist of it seems to be that you want to be able to write
    files in your DSL that can be imported just like a regular Python
    module. Yes, that can be done.

    See PEP 302, Import Hooks:

    http://www.python.org/dev/peps/pep-0302/

    Python's standard importer looks for files with *.py, *.pyc, *.pyd, or
    *.so extensions. You could write an importer that looks for *.dsl
    files, and, instead of loading it as a Python file, invokes your DSL
    parser.


    Carl Banks
     
    Carl Banks, Jan 7, 2009
    #14
  15. On Jan 7, 7:50 am, J Kenneth King <> wrote:
    > Jonathan Gardner <> writes:
    > > On Jan 6, 12:24 pm, J Kenneth King <> wrote:
    > >> Jonathan Gardner <> writes:
    > >> > On Jan 6, 8:18 am, sturlamolden <> wrote:
    > >> >> On Jan 6, 4:32 pm, mark <> wrote:

    >
    > >> >> > I want to implement a internal DSL in Python. I would like the syntax
    > >> >> > as human readable as possible.

    >
    > >> >> Also beware that Python is not Lisp. You cannot define new syntax (yes
    > >> >> I've seen the goto joke).

    >
    > >> > This isn't really true. You can, for instance, write a program (in
    > >> > Python) that takes your pseudo-Python and converts it into Python.
    > >> > This is what a number of templating libraries such as Mako do.

    >
    > >> Which is not even close to being the same.

    >
    > >> Lisp - the program source is also the data format

    >
    > >> Python - the program source is a string

    >
    > >> I could go on a really long rant about how the two are worlds apart, but
    > >> I'll let Google tell you if you're really interested.

    >
    > > I get that Lisp is special because you can hack on the reader as it is
    > > reading the file in. This is strongly discouraged behavior, as far as
    > > I know, despite the number of cute hacks you can accomplish with it.

    >
    > It is generally discouraged unless there's a reason for it.
    >
    > > But consider that this really isn't different than having a program
    > > read in the lisp-with-modification source and spitting out pure lisp,
    > > to be read by an honest-to-gosh lisp program later.

    >
    > > If that's the case, then Lisp and Python really aren't that different
    > > in this regard, except that you don't have the option of modifying the
    > > reader as it reads in the file.

    >
    > I think you are missing the distinction.
    >
    > Lisp expressions are also data structures. A Lisp expression can be
    > passed to functions and macros to be operated on before being
    > executed. When you're writing Lisp source, you're basically looking at
    > the AST on one level and when you start writing macros for your program,
    > you're creating a "DSL" or interface to that AST. Lisp source is
    > eventually expanded to a giant list that is consed by the evaluator (as
    > far as I understand it. I'm just getting into the compiler stuff
    > myself).
    >


    I think you misunderstood what I was trying to explain. Yes, you can
    do those wonderful things with Lisp.

    You can also do wonderful things with Python. Consider programs that
    take some text written in some other language besides Python. Those
    programs interpret and translate the text to Python. Then the programs
    feed the translations to the Python interpreter. Tada! You have a DSL
    in Python.

    No, it's not built in, nor is there any standard, but it is entirely
    possible and people are doing it today. That's how the variety of
    templating solutions work in the Python world. It's why I can write ${x
    +y} in Mako and get a Python program that will do the right thing.

    Alternatively, you can skip the Python interpreter altogether, and
    write your own interpreter for the language. If it's a simple language
    (like the original poster hinted at), this is very easy to do.
     
    Jonathan Gardner, Jan 7, 2009
    #15
  16. On Jan 7, 9:16 am, "Chris Mellon" <> wrote:
    >
    > The OP wants a Ruby-style DSL by which he means "something that lets
    > me write words instead of expressions". The ruby syntax is amenable to
    > this, python (and lisp, for that matter) syntax is not and you can't
    > implement that style of internal DSL in those languages.
    >
    > The answer to the OP is "you can't - use Ruby or modify your requirements".
    >


    As far as putting the code into Python, yeah, you can't put it in
    Python. The best you can do is store it in a string and then interpret
    the string with some function later on.
     
    Jonathan Gardner, Jan 7, 2009
    #16
  17. Re: looking for tips on how to implement "ruby-style" Domain Specific Language in Python

    Jonathan Gardner <> writes:

    > On Jan 7, 9:16 am, "Chris Mellon" <> wrote:
    >>
    >> The OP wants a Ruby-style DSL by which he means "something that lets
    >> me write words instead of expressions". The ruby syntax is amenable to
    >> this, python (and lisp, for that matter) syntax is not and you can't
    >> implement that style of internal DSL in those languages.
    >>
    >> The answer to the OP is "you can't - use Ruby or modify your requirements".
    >>

    >
    > As far as putting the code into Python, yeah, you can't put it in
    > Python. The best you can do is store it in a string and then interpret
    > the string with some function later on.


    That's what I'm saying.

    It seems we're defining "DSL" in two different ways.

    You can't write a DSL in Python because you can't change the syntax and
    you don't have macros.

    You can write a compiler in Python that will compile your "DSL."

    As another poster mentioned, eventually PyPy will be done and then
    you'll get more of an "in-Python" DSL.
     
    J Kenneth King, Jan 8, 2009
    #17
  18. mark

    Kay Schluehr Guest

    On 8 Jan., 16:25, J Kenneth King <> wrote:

    > As another poster mentioned, eventually PyPy will be done and then
    > you'll get more of an "in-Python" DSL.


    May I ask why you consider it as important that the interpreter is
    written in Python? I see no connection between PyPy and syntactical
    Python extensions and the latter isn't an objective of PyPy. You can
    write Python extensions with virtually any Python aware parser.
    M.A.Lemburg already mentioned PLY and PLY is used for Cython. Then
    there is ANTLR which provides a Python grammar. I also know about two
    other Python aware parsers. One of them was written by myself.
     
    Kay Schluehr, Jan 8, 2009
    #18
  19. mark

    mark Guest

    > So you can't make an internal DSL like this that uses Python's built-
    > in grammar.  You'd have to hack the parser or settle for an external
    > preprocessor.


    This time it is really hard for me but I begin accepting the fact that
    I will have to build an external DSL. I experimented some weeks ago
    with ANTLR and the tools work fine but I do not like the extra effort
    to learn and maintain the extra tooling. I think that in the beginning
    the DSL language will have to change a very often as new features are
    added. To implement a standardized rock solid language like SQL ANTLR
    might be the perfect tool but to develop something from scratch that
    will be expanded interactively a internal DSL has huge benefits.

    Please not that I really like ANTLR. It is just the first tool I used
    for this task and I want to double check if other tools fit better to
    my needs.

    I will look into Ply and Pyparsing over the next weeks unless someone
    points out that there is some special tool that makes growing a new
    "fast evolving" language as easy as building an internal DSL. Maybe
    this is all overkill and there is a hacking ruby-style DSLs with
    regular expressions recipe out there? So far I could not find one.

    > However, the gist of it seems to be that you want to be able to write
    > files in your DSL that can be imported just like a regular Python
    > module.  Yes, that can be done.
    >
    > See PEP 302, Import Hooks:
    >
    > http://www.python.org/dev/peps/pep-0302/
    >
    > Python's standard importer looks for files with *.py, *.pyc, *.pyd, or
    > *.so extensions.  You could write an importer that looks for *.dsl
    > files, and, instead of loading it as a Python file, invokes your DSL
    > parser.


    This is really helpful. Thanks for giving me directions.

    Mark
     
    mark, Jan 8, 2009
    #19
  20. Re: looking for tips on how to implement "ruby-style" Domain Specific Language in Python

    Kay Schluehr <> writes:

    > On 8 Jan., 16:25, J Kenneth King <> wrote:
    >
    >> As another poster mentioned, eventually PyPy will be done and then
    >> you'll get more of an "in-Python" DSL.

    >
    > May I ask why you consider it as important that the interpreter is
    > written in Python?


    I don't think it's important for Python to have a meta-circular
    interpreter (though it can't hurt).

    > I see no connection between PyPy and syntactical
    > Python extensions and the latter isn't an objective of PyPy. You can
    > write Python extensions with virtually any Python aware parser.
    > M.A.Lemburg already mentioned PLY and PLY is used for Cython. Then
    > there is ANTLR which provides a Python grammar. I also know about two
    > other Python aware parsers. One of them was written by myself.


    Because... there is no connection to see? I never mentioned any such
    relation.

    DSL's tend to be a natural side-effect of languages which can manipulate
    their own expressions without extensive parsing.

    Creating a new parser that can generate Python AST's is certainly a
    valid approach (and probably the easiest one). It's not the only one.

    It depends on your definition of a DSL.

    My definition isn't satisfied with creating a parser, and so my answers
    reflect that.
     
    J Kenneth King, Jan 8, 2009
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    4
    Views:
    299
    Bruno Desthuilliers
    Aug 6, 2008
  2. Stephan Beal
    Replies:
    13
    Views:
    5,481
    Falcon Kirtaran
    Jan 15, 2009
  3. Replies:
    4
    Views:
    361
    Dutch Masters
    Feb 25, 2009
  4. Matthew Braid
    Replies:
    0
    Views:
    99
    Matthew Braid
    Apr 11, 2006
  5. Replies:
    1
    Views:
    3,538
    Arne Vajhøj
    Jul 31, 2012
Loading...

Share This Page