Re: Compile time evaluation of dictionaries

Discussion in 'Python' started by Terry Reedy, Mar 10, 2011.

  1. Terry Reedy

    Terry Reedy Guest

    On 3/10/2011 11:23 AM, Gerald Britton wrote:
    > Today I noticed that an expression like this:
    >
    > "one:%(one)s two:%(two)s" % {"one": "is the loneliest number", "two":
    > "can be as bad as one"}
    >
    > could be evaluated at compile time, but is not:


    In fact, it could be evaluated at writing time ;-).
    This would be an example of constant folding, except that a dict is not,
    in itself, a constant.

    >>>> dis(compile(

    > ... '"one:%(one)s two:%(two)s" % {"one": "is the loneliest number",
    > "two": "can be as bad as one"}',
    > ... '','exec'))
    > 1 0 LOAD_CONST 0 ('one:%(one)s two:%(two)s')
    > 3 BUILD_MAP 2
    > 6 LOAD_CONST 1 ('is the loneliest number')
    > 9 LOAD_CONST 2 ('one')
    > 12 STORE_MAP
    > 13 LOAD_CONST 3 ('can be as bad as one')
    > 16 LOAD_CONST 4 ('two')
    > 19 STORE_MAP
    > 20 BINARY_MODULO
    > 21 POP_TOP
    > 22 LOAD_CONST 5 (None)
    > 25 RETURN_VALUE
    >>>>

    >
    > Any idea why Python works this way?


    Because that is how the language is defined.

    > I see that, in 3.2, an
    > optimization was done for sets (See "Optimizations" at
    > http://docs.python.org/py3k/whatsnew/3.2.html) though I do not see
    > anything similar for dictionaries.


    I presume you mean this:

    "Python’s peephole optimizer now recognizes patterns such x in {1, 2, 3}
    as being a test for membership in a set of constants. The optimizer
    recasts the set as a frozenset and stores the pre-built constant.

    Now that the speed penalty is gone, it is practical to start writing
    membership tests using set-notation. This style is both semantically
    clear and operationally fast:

    extension = name.rpartition('.')[2]
    if extension in {'xml', 'html', 'xhtml', 'css'}:
    handle(name)

    (Patch and additional tests contributed by Dave Malcolm; issue 6690)."

    This is possible because: sets have a frozen counterpart, and cpython
    *sometimes* follows an 'as if' rule for optimizations. There is no
    frozendict, so the same pattern could not be followed.

    Note first that the optimization is *not* an example of constant
    folding, but recognition that an unbound mutable cannot be mutated, and
    therefore, if all its contents are fixed, can be 'cast' to constant form
    at compile time.

    Note second that the optimization does not apply to all such set
    instances, but only to this particular 'in' context. It follows a
    similar optimization for turning lists into tuples (although in this
    latter case, the programmer could have remembered to use () instead of []).

    The optimization *did* happen because various people started an issue,
    wrote a patch, reviewed the patch, and committed it to the cpython
    source. You might read the issue if you have not:

    http://bugs.python.org/issue6690

    Terry Jan Reedy
     
    Terry Reedy, Mar 10, 2011
    #1
    1. Advertising

  2. On Thu, 10 Mar 2011 17:40:40 -0500, Terry Reedy wrote:

    > On 3/10/2011 11:23 AM, Gerald Britton wrote:
    >> Today I noticed that an expression like this:
    >>
    >> "one:%(one)s two:%(two)s" % {"one": "is the loneliest number", "two":
    >> "can be as bad as one"}
    >>
    >> could be evaluated at compile time, but is not:

    >
    > In fact, it could be evaluated at writing time ;-).


    True, but why do stuff when the compiler can do it for you? Any constant
    folding could be done at writing time, but readability and maintenance
    dictates that we write something like:

    C = 1.0/7

    rather than

    C = 0.14285714285714285


    > This would be an
    > example of constant folding, except that a dict is not, in itself, a
    > constant.


    Nevertheless, since the dict only exists for a single operation, it might
    as well be a constant. In general, Python can't safely make many
    assumptions about compile-time behaviour, since nearly anything can be
    modified at run-time, but the behaviour of built-in literals is one thing
    that can't change.

    I don't see any reason why Python couldn't optimize the above at compile-
    time, and I can only think of two reasons why it won't:

    - lack of interest from anyone willing and able to write a patch;
    - the added complexity may be more than the benefit gained.


    --
    Steven
     
    Steven D'Aprano, Mar 11, 2011
    #2
    1. Advertising

  3. Terry Reedy

    Chris Rebert Guest

    On Thu, Mar 10, 2011 at 4:15 PM, Steven D'Aprano
    <> wrote:
    > On Thu, 10 Mar 2011 17:40:40 -0500, Terry Reedy wrote:
    >> On 3/10/2011 11:23 AM, Gerald Britton wrote:
    >>> Today I noticed that an expression like this:
    >>>
    >>> "one:%(one)s two:%(two)s" % {"one": "is the loneliest number", "two":
    >>> "can be as bad as one"}
    >>>
    >>> could be evaluated at compile time, but is not:

    >>
    >> In fact, it could be evaluated at writing time ;-).

    >
    > True, but why do stuff when the compiler can do it for you?

    <snip>
    > I don't see any reason why Python couldn't optimize the above at compile-
    > time, and I can only think of two reasons why it won't:
    >
    > - lack of interest from anyone willing and able to write a patch;
    > - the added complexity may be more than the benefit gained.


    3. %-formatting is "obsolete and may go away in future versions of Python."
    (See http://docs.python.org/py3k/library/stdtypes.html#old-string-formatting-operations
    )

    Cheers,
    Chris
     
    Chris Rebert, Mar 11, 2011
    #3
  4. On Thu, 10 Mar 2011 16:27:17 -0800, Chris Rebert wrote:


    > 3. %-formatting is "obsolete and may go away in future versions of
    > Python." (See
    > http://docs.python.org/py3k/library/stdtypes.html#old-string-formatting-

    operations
    > )


    There is an awful lot of opposition to that. If it ever happens, it
    probably won't happen until Python4, but even if it happens sooner, you
    could replace

    "one:%(one)s two:%(two)s" % \
    {"one": "is the loneliest number", "two": "can be as bad as one"}

    with the format string equivalent, still using literals, and the same
    constant-folding optimization could occur.

    "one:{one} two:{two}".format(
    **{"one": "is the loneliest number", "two": "can be as bad as one"})

    Admittedly, it would require a significantly smarter peephole optimizer
    to recognise this as a constant, which pushes up the complexity for no
    additional gain, but the principle still applies.



    --
    Steven
     
    Steven D'Aprano, Mar 11, 2011
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. lysdexia
    Replies:
    6
    Views:
    561
    John Machin
    Dec 2, 2007
  2. Brandon
    Replies:
    12
    Views:
    516
    Brandon
    Aug 15, 2008
  3. Carter
    Replies:
    2
    Views:
    529
    Carter
    Mar 4, 2009
  4. Gerald Britton

    Compile time evaluation of dictionaries

    Gerald Britton, Mar 10, 2011, in forum: Python
    Replies:
    2
    Views:
    264
    Steven D'Aprano
    Mar 12, 2011
  5. Gerald Britton

    Compile time evaluation of dictionaries

    Gerald Britton, Mar 14, 2011, in forum: Python
    Replies:
    0
    Views:
    205
    Gerald Britton
    Mar 14, 2011
Loading...

Share This Page