ANN: a mini-language for encapsulating deep-copy operations on Pythondata structures

Discussion in 'Python' started by Steve Howell, Nov 18, 2009.

  1. Steve Howell

    Steve Howell Guest

    During the last few days I have written code in support of a small DDL
    language that encapsulates a concise representation of the
    manipulations needed to make a deep subcopy of a Python-like data
    structure. It is inspired by syntax from mainstream modern languages,
    including, of course, Python. The DDL can be converted to an AST. That
    AST can be walked to either generate Python code that expresses the
    mapping or to generate Python objects that can execute the mapping.
    Either of the prior outputs can then subsequently be applied to real
    world inputs to create new Python data structures from old ones, using
    the mechanisms specified in the original DDL for attribute access,
    dictionary lookup, iteration, method invocation, etc.

    Here is an example of the DDL (and I hate the terminology "DDL," just
    cannot think of anything better):


    {
    'show_table_of_contents',
    'author' {
    .person 'name',
    .person 'location' as city,
    .favorite_books()[
    .title,
    .cost() as expense
    ] as books}
    }


    There are more details here:

    http://showellonprogramming.blogspot.com/2009/11/more-on-python-deep-copy-schema.html

    Apart from shamelessly plugging my blog, I am hoping to generate
    ideas, constructive criticism, etc.

    Feel free to comment here or on the blog. So far there are three
    entries on the blog, all pertaining to Python.

    The implementation of the idea is quite new, but it is a topic that I
    have been pondering for a long time--no matter how expressive your
    programming language of choice might be, there are certain programming
    tasks that just seem needlessly tedious. The mini-language above
    proposes to solve one problem but solve it well.

    I am particularly interested to find out whether somebody has tried
    something like this before.
     
    Steve Howell, Nov 18, 2009
    #1
    1. Advertising

  2. Steve Howell

    Steve Howell Guest

    Re: ANN: a mini-language for encapsulating deep-copy operations onPython data structures

    On Nov 18, 4:34 am, "M.-A. Lemburg" <> wrote:
    > Steve Howell wrote:
    > > [...]
    > > Here is an example of the DDL (and I hate the terminology "DDL," just
    > > cannot think of anything better):

    >
    > >             {
    > >                 'show_table_of_contents',
    > >                 'author' {
    > >                     .person 'name',
    > >                     .person 'location' as city,
    > >                     .favorite_books()[
    > >                         .title,
    > >                         .cost() as expense
    > >                         ] as books}
    > >             }

    >
    > > There are more details here:

    >
    > >http://showellonprogramming.blogspot.com/2009/11/more-on-python-deep-...


    Thanks for your feedback, Marc-Andre! Comments inline...

    >
    > Personally, I find the explicit approach more intuitive, since
    > you immediately see what you are going to get:
    >
    > show_table_of_contents = context['show_table_of_contents']
    > author = context['author']
    > d = {
    >     'show_table_of_contents': show_table_of_contents,
    >     'author': {
    >         'name': author.person['name'],
    >         'city': author.person['location'],
    >         'books': [{
    >             'title': item.title,
    >             'expense': item.cost(),
    >             }
    >             for item in author.favorite_books()],
    >         },
    >     }
    >


    I understand what you mean about the explicit approach. When you go
    for brevity, you sacrifice some explicitness. One of the things I'm
    actually looking to do next is to be able to generate code like what
    you've written above. You could make the generated code even more
    procedural in terms of how it walks the structure--for example,
    replace the list comprehension with a for loop. This could be useful
    for inserting tracing code, for example.

    > I particularly find this part non-intuitive:
    >
    > >                     .favorite_books()[
    > >                         .title,
    > >                         .cost() as expense
    > >                         ] as books}

    >


    Yep, I think what I really want is this:

    .favorite_books()[{
    .title,
    .cost() as expense
    }] as books}

    I am not sure the curlies make the transformation completely
    intuitive, but they are more suggestive that you are generating a list
    of dictionaries.

    > and would leave in the square brackets for __getitem__
    > lookups on these:
    >
    > >                     .person 'name',
    > >                     .person 'location' as city,

    >


    Yep, I agree. The square brackets were not a deliberate omission.
    They were just slightly more tricky to implement because of '[' having
    double syntactical meaning.

    > For additional inspiration, you might want to look at XSLT
    > which provides similar transformations on XML data structures.
    >
    > There are also a number of other transformation languages:
    >
    > http://en.wikipedia.org/wiki/Model_...p://en.wikipedia.org/wiki/Data_transformation
    >


    Thanks for the links!

    > Regarding the term "DDL": that's normally used for "data definition
    > language" and doesn't really have all that much to do with
    > transforming data. You normally define data structures using
    > DDL - without actually putting data into those structures.
    >
    > Why not "PyDTL".... Python data transformation language ?!
    >


    Yep, I like that suggestion. The language does indeed describe a
    transformation. In some ways I was wanting to think of it more
    broadly, in terms that the transformation also acts as a schema for
    any input objects. For example, you could use '{.title, .publisher,
    {.name} }' as a schema to validate that an object behaves like a
    book. I suppose that's still a transformation in an abstract sense,
    where the output is just True, False, or maybe even the slice of the
    transformation that can/cannot be executed.
     
    Steve Howell, Nov 18, 2009
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. VisionSet
    Replies:
    8
    Views:
    4,968
    Tris Orendorff
    Apr 29, 2004
  2. Alex
    Replies:
    2
    Views:
    1,293
  3. Alfonso Morra
    Replies:
    11
    Views:
    755
    Emmanuel Delahaye
    Sep 24, 2005
  4. Replies:
    1
    Views:
    562
    Victor Bazarov
    Jun 24, 2005
  5. Replies:
    26
    Views:
    2,181
    Roland Pibinger
    Sep 1, 2006
Loading...

Share This Page