YIELD_VALUE Byte Code

Discussion in 'Python' started by Andreas Löscher, Nov 11, 2009.

  1. Hi,
    I am not sure if this is the right newsgroup, so if not don't hesitate
    to tell me.

    I am developed a Python to C compiler, so that Byte Code files
    automatically can be translated into C Extension Modules. (And it works
    pretty well --> http://www.coremountains.com/products/bytecoat/)

    While developing, I found something strange concerning the YIELD_VALUE
    OpCode.

    Since Python Version 2.5 it behaves the following:
    1. pop yield value from stack and return it to
    a former gen_send_ex() call from Objects/genobject.c

    2. push the yield value on the stack in gen_send_ex() and return it

    3. when the generator is executed again, the yield value is
    'poped' from the stack again with the POP_TOP opcode

    Now I found that a little strange:
    1. why is the value removed from the stack, than pushed on the
    stack to remove it finally again? the combination of
    YIELD_VALUE and TOP_POP seems hard coded in compile.c
    which means that a TOP_POP follows every YIELD_VALUE
    TOP_POP

    2. If the semantic of the YIELD_VALUE OpCode has changed, why
    is this reached by using another function? Such thing
    should be done in the OpCode.

    (e.a.: instead of retval = POP()
    --> retval = TOP(); Py_INCREF(retval); )


    I am a little confused about this.

    Best,
    Andreas
     
    Andreas Löscher, Nov 11, 2009
    #1
    1. Advertising

  2. Andreas Löscher

    Terry Reedy Guest

    Andreas Löscher wrote:
    > Hi,
    > I am not sure if this is the right newsgroup, so if not don't hesitate
    > to tell me.


    Since there is no CPython internals list, this is the right place to
    start, even though you might end up having to post a slightly off-topic
    query to python-devel just to get the attention of whoever has the
    answer. But there are a few deveoopers who read and post here also.

    > I am developed a Python to C compiler, so that Byte Code files
    > automatically can be translated into C Extension Modules. (And it works
    > pretty well --> http://www.coremountains.com/products/bytecoat/)
    >
    > While developing, I found something strange concerning the YIELD_VALUE
    > OpCode.
    >
    > Since Python Version 2.5 it behaves the following:
    > 1. pop yield value from stack and return it to
    > a former gen_send_ex() call from Objects/genobject.c
    >
    > 2. push the yield value on the stack in gen_send_ex() and return it
    >
    > 3. when the generator is executed again, the yield value is
    > 'poped' from the stack again with the POP_TOP opcode
    >
    > Now I found that a little strange:
    > 1. why is the value removed from the stack, than pushed on the
    > stack to remove it finally again? the combination of
    > YIELD_VALUE and TOP_POP seems hard coded in compile.c
    > which means that a TOP_POP follows every YIELD_VALUE
    > TOP_POP
    >
    > 2. If the semantic of the YIELD_VALUE OpCode has changed, why
    > is this reached by using another function? Such thing
    > should be done in the OpCode.
    >
    > (e.a.: instead of retval = POP()
    > --> retval = TOP(); Py_INCREF(retval); )
    >
    > I am a little confused about this.


    I suspect that what you see reflects the 2.5 change of 'yield x' from a
    statement to an expression to enable gen.send(). It is possible that
    someone did the simplest thing that worked, rather than properly
    rewriting the code. Or maybe the trickery is needed. You can always try
    a simplyfing patch against the test suite and even submit it if it passes.

    Terry Jan Reedy
     
    Terry Reedy, Nov 17, 2009
    #2
    1. Advertising

  3. Andreas Löscher

    Kee Nethery Guest

    Sample CGI for use with Unit Tests

    I've been looking for examples of how to run unit tests on a CGI.
    Didn't find any so I came up with this pared down sample. Don't know
    where to post example code so I figured I'd just email it to this
    list. Perhaps it will be findable by others in the future.

    This CGI with unit test examples takes three separate files to make it
    work. Copy the example code into the files with the names provided and
    put them all into the same directory.

    In my current project, I have lots of unit tests for all the functions
    I have created, but the main CGI code itself had no unit tests and it
    really needed them. In this example, the CGI itself is a handful of
    lines that should never change and the bulk of the processing is in a
    function. This allows me to create unit tests on everything except the
    few unchanging lines that are the generic CGI in the cgiRunner.py file.

    Kee Nethery

    ---------- cgiRunner.py ------------
    #!/usr/bin/env python

    # 2009-11-17

    # this is a simple CGI that is structured to allow for unit tests.

    # this CGI responds to a GET or a POST
    # send it anything and it will parrot it back along with client
    # browser info.

    # This sample has three files:
    # cgiRunner.py this file
    # cgiFunctions.py does the bulk of the CGI processing
    # cgiUnitTest.py the unittest file for this CGI

    # In this example, the bulk of the CGI processing occurs in a
    # function. The function is in a separate file so that the
    # unit tests can be run against it. All this main does is
    # convert the URL parameters into a dictionary and
    # feed that dictionary to the CGI function formatClientData()

    # This is written for Python 2.6.x I'm using 2.6.4 and am trying to use
    # code that works with Python 3.x and up.

    import cgi ## so that I can be a web server CGI
    import cgiFunctions ## where formatClientData() is located

    def main():
    # give me a list of all the user inputs posted to the cgi
    formPostData = cgi.FieldStorage()
    # and turn it into a key value pair dictionary
    # this uses a list comprehension which I modelled after others
    # examples. Normally I'd do a for loop but figured this
    # should be as fast as possible.
    formPostDict = dict((keyValue,formPostData[keyValue].value) \
    for keyValue in formPostData)

    # this is the call to the actual CGI code.
    cgiResponseList = cgiFunctions.formatClientData(formPostDict)

    # I like to return a list from functions so that I can put
    # validity tests in the function. This one basically returns
    # True if the GET or POST had any parameters sent to the
    # CGI, False if none. Just an example.
    if cgiResponseList[1] == True:
    cgiResponseData = cgiResponseList[0]
    else:
    # if no parameters were sent to the CGI, the response
    # describes how to add parameters to the URL that
    # triggers this CGI.
    cgiResponseData = cgiResponseList[0] + '\n\n' + 'The POST \
    or GET you submitted had no user supplied parameters so other than \
    client data. The only data that can be returned is the data \
    provided by your web client. Most CGIs accept parameters and \
    return results based upon those parameters. To provide GET data \
    try adding a parameter like "sample=some_text". A URL with \
    "sample" as a parameter (and sample2 as a second parameter) might \
    look like \n"http://machine.domain.com/cgi-bin/SimpleCgiUnittest.\
    py?sample=some_text&sample2=more_text".'

    # Python 3.x style print statement that works in Python 2.6.x
    print('Content-type: text/html')
    print('')
    print(cgiResponseData)


    main()


    ---------- cgiFunctions.py ------------
    #!/usr/bin/env python

    # 2009-11-17

    import os ## gets the CGI client values like IP and URL

    def formatClientData(formPostDict):
    """
    This function contains the guts of the CGI response.
    It is designed as a function so that I can use
    unittest to test it.
    """
    # create the output variable and then add stuff to it
    cgiResponseData = ''

    # I like to return lists from functions so that more than
    # some kind of validity checking comes back from the
    # function. In this case it's a lame validity check but
    # it's an example.
    hasFormPostData = (len(formPostDict) > 0)

    # for something interesting to return, CGI client data
    cgiResponseData = cgiResponseData + 'from client browser:\n'
    for cgiKey in list(os.environ.keys()):
    # for each client data value, add a line to the output
    cgiResponseData = cgiResponseData + \
    str(cgiKey) + ' = ' + os.environ[cgiKey] + '\n'

    cgiResponseData = cgiResponseData + '\n\nfrom the URL \
    POST or GET:\n\n'
    # cycle through and output the POST or GET inputs
    for keyValuePair in formPostDict:
    cgiResponseData = cgiResponseData + \
    keyValuePair + ' = ' + formPostDict[keyValuePair] + '\n'

    return [cgiResponseData, hasFormPostData]




    ---------- cgiUnitTest.py ------------
    #!/usr/bin/env python

    import cgiFunctions
    import unittest

    class cgiTests(unittest.TestCase):

    ## For debugging of unit tests, to get one unit test
    ## to run first, UPPERCASE the first char of the unittext
    ## name (after "test_") so that it runs first then lower
    ## case it when it should be part of the group.

    def setUp(self):
    # this is for setting external stuff before unit tests
    # this gets called before EVERY test
    pass

    def test_formatClientData_With(self):
    inputDict = {'sample': 'test_text', 'drink': 'tea'}
    cgiResponse = '''from the URL POST or GET:

    sample = test_text
    drink = tea
    '''
    expected0 = True
    expected1 = True
    outputList = cgiFunctions.formatClientData(inputDict)
    outputBoolean0 = (cgiResponse in outputList[0])
    self.assertEqual(outputBoolean0,expected0)
    self.assertEqual(outputList[1],expected1)

    def test_formatClientData_Without(self):
    inputDict = {}
    expected1 = False
    outputList = cgiFunctions.formatClientData(inputDict)
    self.assertEqual(outputList[1],expected1)

    if __name__ == '__main__':
    testSuite=unittest.TestLoader().loadTestsFromTestCase(cgiTests)
    unittest.TextTestRunner(verbosity=2).run(testSuite)
     
    Kee Nethery, Nov 17, 2009
    #3
  4. Andreas Löscher

    greg Guest

    Andreas Löscher wrote:
    > Since Python Version 2.5 it behaves the following:
    > 1. pop yield value from stack and return it to
    > a former gen_send_ex() call from Objects/genobject.c
    >
    > 2. push the yield value on the stack in gen_send_ex() and return


    Are you sure about that? I would expect the value pushed to
    be the one sent back into the generator using a send() call
    if any, or None if the generator is resumed using next()
    rather than send().

    > 3. when the generator is executed again, the yield value is
    > 'poped' from the stack again with the POP_TOP opcode


    This will happen if the result of the yield expression is
    not used for anything in the Python code.

    --
    Greg
     
    greg, Nov 18, 2009
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Andreas
    Replies:
    1
    Views:
    879
    Jonathan Bromley
    May 4, 2004
  2. Bharat Bhushan

    Appending byte[] to another byte[] array

    Bharat Bhushan, Aug 5, 2003, in forum: Java
    Replies:
    15
    Views:
    40,321
    Roedy Green
    Aug 5, 2003
  3. Jean-Daniel Gamache
    Replies:
    0
    Views:
    431
    Jean-Daniel Gamache
    Jul 14, 2004
  4. Peter
    Replies:
    3
    Views:
    734
    Michael Borgwardt
    Aug 5, 2004
  5. gelonida
    Replies:
    1
    Views:
    771
    Gabriel Genellina
    May 6, 2010
Loading...

Share This Page