[OT] code is data

Discussion in 'Python' started by Anton Vredegoor, Jun 17, 2006.

  1. With the inclusion of ElementTree (an XML-parser) in Python25 and recent
    developments concerning JSON (a very Pythonesque but somewhat limited
    XML notation scheme, let's call it statically typed XML) Python seems to
    have reached a stage where it now seems to be possible to completely
    swallow lesser languages code, modify it, and spit out new source code
    targeting the original language the code was written in, or even make a
    translation to other languages.

    The idea is that we now have a fast parser (ElementTree) with a
    reasonable 'API' and a data type (XML or JSON) that can be used as an
    intermediate form to store parsing trees. Especially statically typed
    little languages seem to be very swallow-able. Maybe I will be able to
    reimplement GFABasic (my first love computer language, although not my
    first relationship) someday, just for fun.

    Then there are things like cTypes (calling functions from native DLL's)
    and PyPy (implementing Python in Python).

    All this taken together, to me it starts looking like we're now entering
    a territory that traditionally was exclusively in the Lisp domain.

    Yes, Python had eval and exec for a long time already, and metatypes and
    generators are having some strange unexplored possibilities too, but the
    day will come soon (and at last when PyPy is reaching execution speeds
    close to cPython) where Python will be able to swallow smaller
    languages, and finally it will be able to swallow its own tail, like
    Lisp but then more powerful (because of the widely used standard data
    types and the code exchange between languages that that makes possible).

    Your thoughts please.

    Anton
     
    Anton Vredegoor, Jun 17, 2006
    #1
    1. Advertising

  2. Anton Vredegoor

    John Roth Guest

    Re: code is data

    Anton Vredegoor wrote:
    > With the inclusion of ElementTree (an XML-parser) in Python25 and recent
    > developments concerning JSON (a very Pythonesque but somewhat limited
    > XML notation scheme, let's call it statically typed XML) Python seems to
    > have reached a stage where it now seems to be possible to completely
    > swallow lesser languages code, modify it, and spit out new source code
    > targeting the original language the code was written in, or even make a
    > translation to other languages.


    When I heard of the new AST based compiler, I thought it
    would finally be possible to extend things cleanly. Then
    I learned that they weren't going to allow modification of
    the AST.

    There are a lot of things you can't do in source if the
    language doesn't allow it. Python is pretty good compared
    to other languages, but it's still not possible to create
    new control structures with short circuit semantics.
    And that's one example.

    I saw the "make" statement as a breath of fresh air.
    Then it got shot down for what were, to me, totally
    trivial reasons. That's a second one.

    Sigh.

    John Roth
    >
    >
    > Your thoughts please.
    >
    > Anton
     
    John Roth, Jun 17, 2006
    #2
    1. Advertising

  3. Anton Vredegoor

    Paddy Guest

    Re: code is data

    Anton Vredegoor wrote:
    > With the inclusion of ElementTree (an XML-parser) in Python25 and recent
    > developments concerning JSON (a very Pythonesque but somewhat limited
    > XML notation scheme, let's call it statically typed XML)

    <SNIP
    >
    > Your thoughts please.
    >
    > Anton


    Hi Anton.
    If you mean this JSON: http://www.json.org/example.html
    then I'd just point out that JSON isn't XML-like at all. In fact the
    examples look like valid Python nested dictionaries.

    - Pad.

    P.S. This is good too: http://en.wikipedia.org/wiki/JSON
     
    Paddy, Jun 17, 2006
    #3
  4. Anton Vredegoor

    Ravi Teja Guest

    Re: code is data

    Paddy wrote:
    > Anton Vredegoor wrote:
    > > With the inclusion of ElementTree (an XML-parser) in Python25 and recent
    > > developments concerning JSON (a very Pythonesque but somewhat limited
    > > XML notation scheme, let's call it statically typed XML)

    > <SNIP
    > >
    > > Your thoughts please.
    > >
    > > Anton

    >
    > Hi Anton.
    > If you mean this JSON: http://www.json.org/example.html
    > then I'd just point out that JSON isn't XML-like at all. In fact the
    > examples look like valid Python nested dictionaries.


    It is the same JSON. JSON is typically seen as a human friendly
    replacement for some of the functions that XML is otherwise used for,
    where the full blown XML spec is an overkill and JSON does not need
    complicated parsers in some common languages because it can express
    hierarchical data just like XML.
     
    Ravi Teja, Jun 17, 2006
    #4
  5. Anton Vredegoor

    Ravi Teja Guest

    Re: code is data

    Anton Vredegoor wrote:
    > With the inclusion of ElementTree (an XML-parser) in Python25 and recent
    > developments concerning JSON (a very Pythonesque but somewhat limited
    > XML notation scheme, let's call it statically typed XML) Python seems to
    > have reached a stage where it now seems to be possible to completely
    > swallow lesser languages code, modify it, and spit out new source code
    > targeting the original language the code was written in, or even make a
    > translation to other languages.
    >
    > The idea is that we now have a fast parser (ElementTree) with a
    > reasonable 'API' and a data type (XML or JSON) that can be used as an
    > intermediate form to store parsing trees. Especially statically typed
    > little languages seem to be very swallow-able. Maybe I will be able to
    > reimplement GFABasic (my first love computer language, although not my
    > first relationship) someday, just for fun.
    >
    > Then there are things like cTypes (calling functions from native DLL's)
    > and PyPy (implementing Python in Python).
    >
    > All this taken together, to me it starts looking like we're now entering
    > a territory that traditionally was exclusively in the Lisp domain.
    >
    > Yes, Python had eval and exec for a long time already, and metatypes and
    > generators are having some strange unexplored possibilities too, but the
    > day will come soon (and at last when PyPy is reaching execution speeds
    > close to cPython) where Python will be able to swallow smaller
    > languages, and finally it will be able to swallow its own tail, like
    > Lisp but then more powerful (because of the widely used standard data
    > types and the code exchange between languages that that makes possible).
    >
    > Your thoughts please.


    I don't share your optimism at all. Most of the things you mentioned
    have existed for long. Just because some of them are now included in
    the standard library isn't going to change things drastically.
    Installing them earlier was never hard at all.

    People like to call everything with the lightest semblence, a DSL. That
    gives the feel that the language is more powerful. Ruby people do it
    all the time. Python cannot be called a DSL language until, creating
    them is a natural language feature (like Lisp). And that does not seem
    to be happening anytime soon. Boo for example allows you to write new
    constructs with it's AST library. It still cannot be called a DSL
    "language".

    People have however written various language interpreters (Scheme,
    Forth and yes, even Basic) in Python, just for kicks. Still does not
    make it a DSL language anymore than it makes C a DSL language.

    At present, the closest thing to writing a DSL in Python is Logix
    http://livelogix.net/logix/
    Too bad though, the project is defunct and there has never been enough
    interest in it.

    Personally, I would like to see macros in Python (actually Logix
    succeeding is good enough). But I am no language designer and the
    community has no interest in it. When I absolutely need macros, I will
    go elsewhere.
     
    Ravi Teja, Jun 17, 2006
    #5
  6. Re: code is data

    > Personally, I would like to see macros in Python (actually Logix
    > succeeding is good enough). But I am no language designer and the
    > community has no interest in it. When I absolutely need macros, I will
    > go elsewhere.


    One must wonder, when is that? When do you absolutely need macros?

    --
    mvh Björn
     
    =?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=, Jun 18, 2006
    #6
  7. Anton Vredegoor

    Ravi Teja Guest

    Re: code is data

    BJörn Lindqvist wrote:
    > > Personally, I would like to see macros in Python (actually Logix
    > > succeeding is good enough). But I am no language designer and the
    > > community has no interest in it. When I absolutely need macros, I will
    > > go elsewhere.

    >
    > One must wonder, when is that? When do you absolutely need macros?


    Whenever there is significant boiler plate code that functions and
    classes cannot eliminate alone.
    Whenever there is a more elegant way to express your code.

    Python 2.5 introduced conditional expressions and with statement. With
    macros, one would not have to wait for the language team to implement
    them. More so for features which only a small part of the community has
    an interest in.

    I *like* 1..5 (ada, ruby) instead of range(5). If I had macros, I would
    have done it myself for *my* code.

    I would like special behaviour code blocks in my programs, for say DBC
    (I am aware of the work arounds).
     
    Ravi Teja, Jun 18, 2006
    #7
  8. Anton Vredegoor

    Paddy Guest

    Re: code is data

    Ravi Teja wrote:
    > BJörn Lindqvist wrote:
    > > > Personally, I would like to see macros in Python (actually Logix
    > > > succeeding is good enough). But I am no language designer and the
    > > > community has no interest in it. When I absolutely need macros, I will
    > > > go elsewhere.

    > >
    > > One must wonder, when is that? When do you absolutely need macros?

    >
    > Whenever there is significant boiler plate code that functions and
    > classes cannot eliminate alone.
    > Whenever there is a more elegant way to express your code.
    >


    Me, I am torn. I should now better. I have listened to the arguments
    against Macros in Python and the ones that struck home were the
    argument for maintainability:
    Without macros, Python is Python. Statements do what you expect.

    And the argument against DSLs altogether:
    Make Python your DSL! If you design your own DSL before long you start
    to embellish it with more statements or datatypes and before long it
    becomes complex. If you used Python from the beginning then you would
    have a community for support.

    I know the arguments, but every once in a while I think if only I could
    craft my own ??? statement or ....

    Don't go their Paddy.

    ;-)
     
    Paddy, Jun 18, 2006
    #8
  9. Anton Vredegoor

    Ravi Teja Guest

    Re: code is data

    Paddy wrote:
    > Ravi Teja wrote:
    > > BJörn Lindqvist wrote:
    > > > > Personally, I would like to see macros in Python (actually Logix
    > > > > succeeding is good enough). But I am no language designer and the
    > > > > community has no interest in it. When I absolutely need macros, I will
    > > > > go elsewhere.
    > > >
    > > > One must wonder, when is that? When do you absolutely need macros?

    > >
    > > Whenever there is significant boiler plate code that functions and
    > > classes cannot eliminate alone.
    > > Whenever there is a more elegant way to express your code.
    > >

    >
    > Me, I am torn. I should now better. I have listened to the arguments
    > against Macros in Python and the ones that struck home were the
    > argument for maintainability:
    > Without macros, Python is Python. Statements do what you expect.


    Yes! I heard those arguments too. And I am not convinced.

    Static language programmer: Lack of static typing removes the necessary
    safeguards. The code is more error prone. Objects have behavior that is
    not obvious.
    Dynamic language programmer: Really? I don't seem to have any more bugs
    than in my statically typed code. And my code is compact and reads
    better. I don't want to go back.

    No to macros proponent: Macros introduce a lot of potential for abuse.
    Code will be worse to read than Perl.
    Macros proponent: Really? We have been doing macros for decades. We all
    think our code is better for macros, not worse. We are not going back.

    I just don't get it. Don't we often invoke the "We are all adults here"
    argument.

    Writing a macro is not as simple as writing a function. Sort of like
    metaclasses. Many will stay off them. Those that really need them will
    walk that extra mile. Don't we all believe that "Simple should be
    possible. Complex should be doable"

    > And the argument against DSLs altogether:
    > Make Python your DSL! If you design your own DSL before long you start
    > to embellish it with more statements or data types and before long it
    > becomes complex. If you used Python from the beginning then you would
    > have a community for support.


    Python has a low cognitive overhead. But it not a DSL by definition. No
    language can be. The idea is that when the domain changes, a DSL should
    be driven by the new domain as warranted. In other words, driven "by
    the problem, not the tool".

    I don't want "a DSL". I want a language that allows me to make "my
    DSL" based on it. That means I don't loose the community connection. I
    can still use all the rich libraries in my DSL.

    I like Python for its indentation syntax, sensible semantics and
    readability. I invested a lot of time in Python. After much language
    hopping, I settled with Python. I like the community and the code base
    available for it. The libraries just seem to be designed at the right
    level of abstraction for me (as opposed to say, Java). When I need to
    do something, I know where to go. But all this ties me to the language
    tightly that I cannot change.

    > I know the arguments, but every once in a while I think if only I could
    > craft my own ??? statement or ....


    My thoughts exactly.

    Web frameworks, which seem to be the rage now in Python community could
    have benefited tremendously from Macro capabilities since they have a
    lot of boiler plate.
     
    Ravi Teja, Jun 18, 2006
    #9
  10. Re: code is data

    Ravi Teja wrote:

    > Web frameworks, which seem to be the rage now in Python community could
    > have benefited tremendously from Macro capabilities since they have a
    > lot of boiler plate.


    they do? methinks you haven't done much web programming lately...

    </F>
     
    Fredrik Lundh, Jun 18, 2006
    #10
  11. Anton Vredegoor

    Ravi Teja Guest

    Re: code is data

    Fredrik Lundh wrote:
    > Ravi Teja wrote:
    >
    > > Web frameworks, which seem to be the rage now in Python community could
    > > have benefited tremendously from Macro capabilities since they have a
    > > lot of boiler plate.

    >
    > they do? methinks you haven't done much web programming lately...
    >
    > </F>


    You blogged on Django. Let's use that. Don't you think model creation
    in Django can be represented better, given that it is done often
    enough?

    Let's take an example from the official tutorial
    from
    http://www.djangoproject.com/documentation/tutorial1/#creating-models

    class Poll(models.Model):
    question = models.CharField(maxlength=200)
    pub_date = models.DateTimeField('date published')

    class Choice(models.Model):
    poll = models.ForeignKey(Poll)
    choice = models.CharField(maxlength=200)
    votes = models.IntegerField()

    I don't use Django and I made this up quickly, so please don't pick on
    subtleties.

    @Poll:
    question: char length 200
    pub_date('date published'): date

    @Choice:
    poll -> Poll
    choice: char length 200
    votes: int

    The following is my rationale. Annoted variables, symbols and code
    layout visually cue more efficiently to the object nature than do
    explicit text definitions. Of course, this is only sensible when there
    aren't too many of any of those. In that case, the cognitive cost of
    notation outweighs the representational cost of text.

    Representational minimalism is troublesome in general code (ala Perl),
    but not so in a DSL where the context is constrained.

    I would also like to symbolize field types since they occur so commonly
    in a definition file and only a few of them are commonly used. I admit
    though that I find the code below a bit visually jarring and I might
    use something else. But it serves to illustrate the point. I chose the
    respective symbols based on their colloquial use and association with
    the field types.

    @Poll:
    $question: length 200
    %pub_date('date published')

    @Choice:
    poll -> Poll
    $choice: length 200
    #votes

    Since you are on thread and are a prominent and involved member of the
    Python community, I would like it if you (or any such other) can
    provide feedback on the rest of my previous post rather than be
    dismissive by just a small portion of it. Perhaps, that will give me
    some insight how these language design decisions are rationally made (I
    am not strictly a programmer by profession, much less a language
    designer).
     
    Ravi Teja, Jun 18, 2006
    #11
  12. Re: code is data

    Ravi Teja <> said:
    > I *like* 1..5 (ada, ruby) instead of range(5). If I had macros, I would
    > have done it myself for *my* code.


    You can write your own preprocessor to handle things like that.
    --
    Roberto Bonvallet
     
    Roberto Bonvallet, Jun 18, 2006
    #12
  13. Re: code is data

    > > > community has no interest in it. When I absolutely need macros, I will
    > > > go elsewhere.

    > I *like* 1..5 (ada, ruby) instead of range(5). If I had macros, I would
    > have done it myself for *my* code.


    I think this example more is a symptom of a childish need to get
    things your way than of a deficiency in Python. BTW, range(5) = 0..4
    in Ada and Ruby.

    You said "when I absolutely need macros" but none of your examples
    demonstrate any "absolute need." I can't see your point.

    --
    mvh Björn
     
    =?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=, Jun 18, 2006
    #13
  14. Anton Vredegoor wrote:
    > With the inclusion of ElementTree (an XML-parser) in Python25 and recent
    > developments concerning JSON (a very Pythonesque but somewhat limited
    > XML notation scheme, let's call it statically typed XML)


    JSON stands for JavaScript Object Notation, and has *nothing* to do with
    XML - except for the fact that it's more and more used instead of XML
    for AJAX stuff.

    > Python seems to
    > have reached a stage where it now seems to be possible to completely
    > swallow lesser languages code, modify it, and spit out new source code
    > targeting the original language the code was written in, or even make a
    > translation to other languages.


    If you mean "parsing source in a given format and outputting another -
    modified or not - representation, in the same or another format", Python
    as always been able to do so.

    > The idea is that we now have a fast parser (ElementTree) with a
    > reasonable 'API' and a data type (XML or JSON) that can be used as an
    > intermediate form to store parsing trees. Especially statically typed
    > little languages seem to be very swallow-able. Maybe I will be able to
    > reimplement GFABasic (my first love computer language, although not my
    > first relationship) someday, just for fun.
    >
    > Then there are things like cTypes (calling functions from native DLL's)
    > and PyPy (implementing Python in Python).
    >
    > All this taken together, to me it starts looking like we're now entering
    > a territory that traditionally was exclusively in the Lisp domain.


    Sorry, but I just don't get the point. Parsing, working with trees and
    calling native code are in no way "exclusively in the Lisp domain".

    > Yes, Python had eval and exec for a long time already, and metatypes and
    > generators are having some strange unexplored possibilities too, but the
    > day will come soon (and at last when PyPy is reaching execution speeds
    > close to cPython) where Python will be able to swallow smaller
    > languages, and finally it will be able to swallow its own tail, like
    > Lisp but then more powerful


    I'm afraid Python is still very far from Lisp - and will never get there
    (FWIW, this seems not to be the goal anyway).

    > (because of the widely used standard data
    > types and the code exchange between languages that that makes possible).


    I still don't get the point.

    > Your thoughts please.
    >
    > Anton



    --
    bruno desthuilliers
    python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
    p in ''.split('@')])"
     
    bruno at modulix, Jun 19, 2006
    #14
  15. Re: code is data

    BJörn Lindqvist wrote:
    >> Personally, I would like to see macros in Python (actually Logix
    >> succeeding is good enough). But I am no language designer and the
    >> community has no interest in it. When I absolutely need macros, I will
    >> go elsewhere.

    >
    >
    > One must wonder, when is that? When do you absolutely need macros?
    >

    One must wonder, when do you absolutely need HOFs, closures, OO,
    functions, or even structured programming. All we 'absolutely' need is
    tests and gotos... (and love, of course !-).

    --
    bruno desthuilliers
    python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
    p in ''.split('@')])"
     
    bruno at modulix, Jun 19, 2006
    #15
  16. Re: code is data

    Ravi Teja wrote:
    > BJörn Lindqvist wrote:
    >
    >>>Personally, I would like to see macros in Python (actually Logix
    >>>succeeding is good enough). But I am no language designer and the
    >>>community has no interest in it. When I absolutely need macros, I will
    >>>go elsewhere.

    >>
    >>One must wonder, when is that? When do you absolutely need macros?

    >
    >
    > Whenever there is significant boiler plate code that functions and
    > classes cannot eliminate alone.
    > Whenever there is a more elegant way to express your code.
    >
    > Python 2.5 introduced conditional expressions and with statement. With
    > macros, one would not have to wait for the language team to implement
    > them. More so for features which only a small part of the community has
    > an interest in.
    >
    > I *like* 1..5 (ada, ruby) instead of range(5). If I had macros, I would
    > have done it myself for *my* code.


    And that's the downside with macros - and with anything that's not
    officially part of the language or it's standard lib : everybody
    implements it it's own way, and you end up with dozens non-standard ways
    of doing the same thing.

    Not to say this is absolutely bad, but there's a balance to be found
    here. One could do function decorators long before we had official
    syntactic sugar for it, but it only started to be a common idiom with
    the @decorator syntax. Python 2.5 introduces a 'partial' type, that is
    quite easy to implement with 2.4 (and probably with older versions too),
    but having it in the builtins or standard lib means it will become the
    standard way to do it - no need to deal with half a dozen half-backed
    implementations of it no more.


    --
    bruno desthuilliers
    python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
    p in ''.split('@')])"
     
    bruno at modulix, Jun 19, 2006
    #16
  17. Re: code is data

    Ravi Teja wrote:

    (snip)
    > Annoted variables, symbols and code
    > layout visually cue more efficiently to the object nature than do
    > explicit text definitions. Of course, this is only sensible when there
    > aren't too many of any of those. In that case, the cognitive cost of
    > notation outweighs the representational cost of text.
    >
    > Representational minimalism is troublesome in general code (ala Perl),
    > but not so in a DSL where the context is constrained.


    This still impose the need to learn a new language.

    > I would also like to symbolize field types since they occur so commonly
    > in a definition file and only a few of them are commonly used. I admit
    > though that I find the code below a bit visually jarring and I might
    > use something else. But it serves to illustrate the point. I chose the
    > respective symbols based on their colloquial use and association with
    > the field types.
    >
    > @Poll:
    > $question: length 200
    > %pub_date('date published')
    >
    > @Choice:
    > poll -> Poll
    > $choice: length 200
    > #votes


    <IMHO>Yuck</IMHO>.


    --
    bruno desthuilliers
    python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
    p in ''.split('@')])"
     
    bruno at modulix, Jun 19, 2006
    #17
  18. bruno at modulix wrote:

    > I still don't get the point.


    Well, I've got to be careful here, lest I'd be associated with the
    terr.., eh, the childp..., eh the macro-enablers.

    The idea is to have a way to transform a Python (.py) module into XML
    and then do source code manipulations in XML-space using ElementTree.

    But rest assured, there is no such module, nor will we ever need it for
    anything.

    Anton

    "use cases are for the faint-hearted"
     
    Anton Vredegoor, Jun 19, 2006
    #18
  19. Re: code is data

    John Roth wrote:
    > I saw the "make" statement as a breath of fresh air.
    > Then it got shot down for what were, to me, totally
    > trivial reasons.


    Which reasons? I as I recall, Guido cut it out without giving any
    reason.
    Of course Guido has the right to do so, but it is not respectful of
    all the work people like Steven Bethard and others did :-(

    Michele Simionato
     
    Michele Simionato, Jun 19, 2006
    #19
  20. Anton Vredegoor wrote:
    > bruno at modulix wrote:
    >
    >> I still don't get the point.

    >
    >
    > Well, I've got to be careful here, lest I'd be associated with the
    > terr.., eh, the childp..., eh the macro-enablers.
    >
    > The idea is to have a way to transform a Python (.py) module into XML
    > and then do source code manipulations in XML-space using ElementTree.


    My my my... I'm not against the idea of dynamic source code
    transformation, but for heaven's sake, *why* would one put XML in the
    mix ???????


    --
    bruno desthuilliers
    python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
    p in ''.split('@')])"
     
    bruno at modulix, Jun 19, 2006
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. =?Utf-8?B?Q2FybG8gTWFyY2hlc29uaQ==?=

    Fire Code behind code AND Javascript code associated to a Button Click Event

    =?Utf-8?B?Q2FybG8gTWFyY2hlc29uaQ==?=, Feb 10, 2004, in forum: ASP .Net
    Replies:
    4
    Views:
    21,245
    =?Utf-8?B?Q2FybG8gTWFyY2hlc29uaQ==?=
    Feb 11, 2004
  2. Author Tarun Tyagi
    Replies:
    0
    Views:
    718
    Author Tarun Tyagi
    Dec 29, 2004
  3. Bobby Edward

    How add data to Entity Data Model via code?

    Bobby Edward, Sep 23, 2008, in forum: ASP .Net
    Replies:
    0
    Views:
    395
    Bobby Edward
    Sep 23, 2008
  4. A. Atienza

    Passing code/data from code-behind to ASPX page

    A. Atienza, Dec 8, 2004, in forum: ASP .Net Datagrid Control
    Replies:
    1
    Views:
    320
    Eliyahu Goldin
    Dec 8, 2004
  5. aratienza

    Passing data/code from code-behind to aspx

    aratienza, Dec 8, 2004, in forum: ASP .Net Web Controls
    Replies:
    1
    Views:
    117
    haile
    Dec 10, 2004
Loading...

Share This Page