Extending Python by Adding Keywords & Data types

Discussion in 'Python' started by Maximus Decimus, Jul 31, 2007.

  1. HI all,

    I am using python v2.5 and I am an amateur working on python. I am
    extending python for my research work and would like some help and
    guidance w.r.t this matter from you experienced python developers.

    II want to add some more KEYWORDS and DATATYPES into the python script
    apart from the existing ones.

    It would be really great if anybody could guide me as which files and
    modules I need to look to make these additions and enhancements to the
    existing python version. Thanks a lot in advance and your help is
    really appreciated.

    Vishak
    Maximus Decimus, Jul 31, 2007
    #1
    1. Advertising

  2. On Tue, 31 Jul 2007 20:28:59 +0000, Maximus Decimus wrote:

    > II want to add some more KEYWORDS and DATATYPES into the python script
    > apart from the existing ones.


    New data types are easy: Just implement the classes.

    Why do you need new keywords? Can't the problem at hand be expressed in
    functions and classes/methods?

    Ciao,
    Marc 'BlackJack' Rintsch
    Marc 'BlackJack' Rintsch, Jul 31, 2007
    #2
    1. Advertising

  3. Since, I am an amateur in using python, could you please be more
    specific. For new data types, you had asked to implement the classes.
    I intend to use C for implementing these data types. So where do i
    need to implement these classes ie under which file or module in the
    python package.

    Thanks in advance.


    On Jul 31, 5:16 pm, Marc 'BlackJack' Rintsch <> wrote:
    > On Tue, 31 Jul 2007 20:28:59 +0000, Maximus Decimus wrote:
    > > II want to add some more KEYWORDS and DATATYPES into the python script
    > > apart from the existing ones.

    >
    > New data types are easy: Just implement the classes.
    >
    > Why do you need new keywords? Can't the problem at hand be expressed in
    > functions and classes/methods?
    >
    > Ciao,
    > Marc 'BlackJack' Rintsch
    Maximus Decimus, Aug 1, 2007
    #3
  4. Maximus Decimus

    Ben Finney Guest

    Maximus Decimus <> writes:

    > II want to add some more KEYWORDS


    Adding keywords can only be done by changing the parser for the Python
    compiler. You can download the source for your preferred
    implementation of Python (e.g. CPython) and change it there.

    But likely that's not the best approach. Can you tell us what problem
    you are trying to solve, and why you think adding keywords to the
    language is desirable?

    > and DATATYPES into the python script apart from the existing ones.


    Adding datatypes is done by defining classes. Please work through the
    tutorial (as a series of exercises to perform and understand) in order
    to get a better grounding in these and other basic topics of Python.

    <URL:http://docs.python.org/tut/>

    --
    \ "Smoking cures weight problems. Eventually." -- Steven Wright |
    `\ |
    _o__) |
    Ben Finney
    Ben Finney, Aug 1, 2007
    #4
  5. On Wed, 01 Aug 2007 01:42:38 +0000, Maximus Decimus wrote:

    > Since, I am an amateur in using python, could you please be more
    > specific. For new data types, you had asked to implement the classes.
    > I intend to use C for implementing these data types. So where do i
    > need to implement these classes ie under which file or module in the
    > python package.


    Like Ben Finney said, first do something about the amateur status and work
    through the tutorial.

    Everything in Python is an object. And classes are a way to define
    templates for new objects. After you know classes better, ask yourself
    how you would implement the already existing data types with classes
    yourself and then you'll get an idea how to create new data types.

    If you want to implement new types in C there's a tutorial in the docs
    called “Extending and Embedding†and a reference of the Python/C API.

    But maybe you should start with implementing them in Python and only
    convert those parts to C code that are *really* to slow in the prototype.
    Really means measure and don't guess.

    A nice step between Python and C is Pyrex, a Python-like language with some
    restrictions but also C data types. Pyrex is then translated to C and can
    be compiled as Python extension module.

    Ciao,
    Marc 'BlackJack' Rintsch
    Marc 'BlackJack' Rintsch, Aug 1, 2007
    #5
  6. Maximus Decimus wrote:

    > Since, I am an amateur in using python,


    <http://en.wikipedia.org/wiki/Amateur>
    | Most commonly an amateur is understood to be someone who does
    | something without pay or formal training. Conversely, a
    | professional is someone who has received training in a particular
    | area and who also makes a living from it.

    So, you're not being paid coding in Python, and/or have
    no "official" training. Many readers of this list share this
    situation, me too.

    > could you please be more specific.


    I return the question. Your statements are most vague. Please give
    an example of what you try to realise.

    > For new data types, you had asked to implement the classes. I
    > intend to use C for implementing these data types. So where do i
    > need to implement these classes ie under which file or module in
    > the python package.


    Please do yourself a favor and read. Feel free to ask again for
    specific problems.

    http://docs.python.org/tut/node3.html
    http://docs.python.org/tut/node11.html

    Regards,


    Björn
    --
    BOFH excuse #135:

    You put the disk in upside down.
    Bjoern Schliessmann, Aug 1, 2007
    #6
  7. Maximus Decimus

    Paul McGuire Guest

    On Jul 31, 3:28 pm, Maximus Decimus <> wrote:
    >
    > I am using python v2.5 and I am an amateur working on python. I am
    > extending python for my research work and would like some help and
    > guidance w.r.t this matter from you experienced python developers.
    >
    > II want to add some more KEYWORDS and DATATYPES into the python script
    > apart from the existing ones.
    >


    Vishak -

    Let me echo the sentiments of the other posters who replied to your
    message. It is *very* unusual for a new Python user to begin their
    Pythonic journey by modifying the Python interpreter to add new
    keywords and datatypes. Python is designed to be a very flexible and
    extensible language just as it is.

    Since you are familiar with C, your question strikes those of us on
    this group as one who would write "I'm just learning C and I would
    like to modify the core library to add some new functions for my
    research." And all the readers on comp.lang.c scratch their heads,
    thinking "Huh? Why doesn't this guy just write the functions in his
    own program, the way the language was designed for him to do in the
    first place?"

    Now maybe your research is in the field of language design, and has
    something to do with the relative ease/difficulty of modifying
    computer languages. Then it would make sense for you to dive right in
    to learning how to modify Python's compiler.

    But if your research was in just about any other field (from finite
    elements analysis to genetic programming to genetics to simulations to
    process control to whatever), DON'T start by modifying Python - start
    by LEARNING Python.

    When a new datatype is required in a Python program, then the
    programmer writes a class to implement this datatype. The class
    itself is written in Python, but it can be used just like any built-in
    datatype. For instance, here is a new datatype I just thought up - a
    Box that can hold up to 'n' objects.

    class Box(object):
    def __init__(self,n):
    self.capacity = n
    self.contents = []

    def add(self,other):
    if len(self.contents) < self.capacity:
    self.contents.append( other )
    else:
    raise ValueError("can't add any more to this Box")

    def __iadd__(self,other):
    self.add(other)
    return self

    box = Box(3)

    # add stuff to the box until it overflows
    while(True):
    box += object()


    Voila! I created a new datatype, Box, and even added support for it
    to understand how to use the '+=' operator so that adding objects to
    the Box looks like an addition statement. All without modifying
    Python itself. (Here's an exercise - extend this example so that the
    Box has a limited capacity in the *weight* of the added objects, as
    well as in the number of objects.)

    This is the basic way that one extends Python with their own new
    datatypes and methods. New keywords are a little rarer, but really,
    start by just adding methods, like the add method above. Python users
    around the world develop a wide range of applications and programs
    using just these techniques, and never touch the Python compiler
    itself.

    And here is a case you'd like to avoid. Let's say you start by
    learning how to modify Python because you need a general-purpose
    container for things. You spend two weeks learning how to do this,
    getting your code mostly debugged, and then you post to
    comp.lang.python your proud achievement. Immediately the replies come
    back, "Congratulations, newbie, you just reinvented the built-in list
    type." Without LEARNING Python, you wont know what is already
    provided in the language.

    So, in general this is a helpful group, and it is possible that you DO
    need to learn how to add datatypes and keywords to Python as your
    first objective. We're not trying to pry, but give us a bit more
    detail. Someone might even rough out for you what one of your new
    datatypes might look like. So pray, tell us what sort of specialized
    datatypes and keywords do you think you need to add, and we'll try to
    point you in one or more directions.

    -- Paul
    Paul McGuire, Aug 1, 2007
    #7
  8. On Aug 1, 9:08 am, Paul McGuire <> wrote:
    > On Jul 31, 3:28 pm, Maximus Decimus <> wrote:
    >
    >
    >
    > > I am using python v2.5 and I am an amateur working on python. I am
    > > extending python for my research work and would like some help and
    > > guidance w.r.t this matter from you experienced python developers.

    >
    > > II want to add some more KEYWORDS and DATATYPES into the python script
    > > apart from the existing ones.

    >
    > Vishak -
    >
    > Let me echo the sentiments of the other posters who replied to your
    > message. It is *very* unusual for a new Python user to begin their
    > Pythonic journey by modifying the Python interpreter to add new
    > keywords and datatypes. Python is designed to be a very flexible and
    > extensible language just as it is.
    >
    > Since you are familiar with C, your question strikes those of us on
    > this group as one who would write "I'm just learning C and I would
    > like to modify the core library to add some new functions for my
    > research." And all the readers on comp.lang.c scratch their heads,
    > thinking "Huh? Why doesn't this guy just write the functions in his
    > own program, the way the language was designed for him to do in the
    > first place?"
    >
    > Now maybe your research is in the field of language design, and has
    > something to do with the relative ease/difficulty of modifying
    > computer languages. Then it would make sense for you to dive right in
    > to learning how to modify Python's compiler.
    >
    > But if your research was in just about any other field (from finite
    > elements analysis to genetic programming to genetics to simulations to
    > process control to whatever), DON'T start by modifying Python - start
    > by LEARNING Python.
    >
    > When a new datatype is required in a Python program, then the
    > programmer writes a class to implement this datatype. The class
    > itself is written in Python, but it can be used just like any built-in
    > datatype. For instance, here is a new datatype I just thought up - a
    > Box that can hold up to 'n' objects.
    >
    > class Box(object):
    > def __init__(self,n):
    > self.capacity = n
    > self.contents = []
    >
    > def add(self,other):
    > if len(self.contents) < self.capacity:
    > self.contents.append( other )
    > else:
    > raise ValueError("can't add any more to this Box")
    >
    > def __iadd__(self,other):
    > self.add(other)
    > return self
    >
    > box = Box(3)
    >
    > # add stuff to the box until it overflows
    > while(True):
    > box += object()
    >
    > Voila! I created a new datatype, Box, and even added support for it
    > to understand how to use the '+=' operator so that adding objects to
    > the Box looks like an addition statement. All without modifying
    > Python itself. (Here's an exercise - extend this example so that the
    > Box has a limited capacity in the *weight* of the added objects, as
    > well as in the number of objects.)
    >
    > This is the basic way that one extends Python with their own new
    > datatypes and methods. New keywords are a little rarer, but really,
    > start by just adding methods, like the add method above. Python users
    > around the world develop a wide range of applications and programs
    > using just these techniques, and never touch the Python compiler
    > itself.
    >
    > And here is a case you'd like to avoid. Let's say you start by
    > learning how to modify Python because you need a general-purpose
    > container for things. You spend two weeks learning how to do this,
    > getting your code mostly debugged, and then you post to
    > comp.lang.python your proud achievement. Immediately the replies come
    > back, "Congratulations, newbie, you just reinvented the built-in list
    > type." Without LEARNING Python, you wont know what is already
    > provided in the language.
    >
    > So, in general this is a helpful group, and it is possible that you DO
    > need to learn how to add datatypes and keywords to Python as your
    > first objective. We're not trying to pry, but give us a bit more
    > detail. Someone might even rough out for you what one of your new
    > datatypes might look like. So pray, tell us what sort of specialized
    > datatypes and keywords do you think you need to add, and we'll try to
    > point you in one or more directions.
    >
    > -- Paul


    On Aug 1, 9:08 am, Paul McGuire <> wrote:
    > On Jul 31, 3:28 pm, Maximus Decimus <> wrote:
    >
    >
    >
    > > I am using python v2.5 and I am an amateur working on python. I am
    > > extending python for my research work and would like some help and
    > > guidance w.r.t this matter from you experienced python developers.

    >
    > > II want to add some more KEYWORDS and DATATYPES into the python script
    > > apart from the existing ones.

    >
    > Vishak -
    >
    > Let me echo the sentiments of the other posters who replied to your
    > message. It is *very* unusual for a new Python user to begin their
    > Pythonic journey by modifying the Python interpreter to add new
    > keywords and datatypes. Python is designed to be a very flexible and
    > extensible language just as it is.
    >
    > Since you are familiar with C, your question strikes those of us on
    > this group as one who would write "I'm just learning C and I would
    > like to modify the core library to add some new functions for my
    > research." And all the readers on comp.lang.c scratch their heads,
    > thinking "Huh? Why doesn't this guy just write the functions in his
    > own program, the way the language was designed for him to do in the
    > first place?"
    >
    > Now maybe your research is in the field of language design, and has
    > something to do with the relative ease/difficulty of modifying
    > computer languages. Then it would make sense for you to dive right in
    > to learning how to modify Python's compiler.
    >
    > But if your research was in just about any other field (from finite
    > elements analysis to genetic programming to genetics to simulations to
    > process control to whatever), DON'T start by modifying Python - start
    > by LEARNING Python.
    >
    > When a new datatype is required in a Python program, then the
    > programmer writes a class to implement this datatype. The class
    > itself is written in Python, but it can be used just like any built-in
    > datatype. For instance, here is a new datatype I just thought up - a
    > Box that can hold up to 'n' objects.
    >
    > class Box(object):
    > def __init__(self,n):
    > self.capacity = n
    > self.contents = []
    >
    > def add(self,other):
    > if len(self.contents) < self.capacity:
    > self.contents.append( other )
    > else:
    > raise ValueError("can't add any more to this Box")
    >
    > def __iadd__(self,other):
    > self.add(other)
    > return self
    >
    > box = Box(3)
    >
    > # add stuff to the box until it overflows
    > while(True):
    > box += object()
    >
    > Voila! I created a new datatype, Box, and even added support for it
    > to understand how to use the '+=' operator so that adding objects to
    > the Box looks like an addition statement. All without modifying
    > Python itself. (Here's an exercise - extend this example so that the
    > Box has a limited capacity in the *weight* of the added objects, as
    > well as in the number of objects.)
    >
    > This is the basic way that one extends Python with their own new
    > datatypes and methods. New keywords are a little rarer, but really,
    > start by just adding methods, like the add method above. Python users
    > around the world develop a wide range of applications and programs
    > using just these techniques, and never touch the Python compiler
    > itself.
    >
    > And here is a case you'd like to avoid. Let's say you start by
    > learning how to modify Python because you need a general-purpose
    > container for things. You spend two weeks learning how to do this,
    > getting your code mostly debugged, and then you post to
    > comp.lang.python your proud achievement. Immediately the replies come
    > back, "Congratulations, newbie, you just reinvented the built-in list
    > type." Without LEARNING Python, you wont know what is already
    > provided in the language.
    >
    > So, in general this is a helpful group, and it is possible that you DO
    > need to learn how to add datatypes and keywords to Python as your
    > first objective. We're not trying to pry, but give us a bit more
    > detail. Someone might even rough out for you what one of your new
    > datatypes might look like. So pray, tell us what sort of specialized
    > datatypes and keywords do you think you need to add, and we'll try to
    > point you in one or more directions.
    >
    > -- Paul



    Firstly to Paul, I clearly got your reply and I am working hard to
    know more about python by going through the tutorials. I did start
    with the doc tutorial available along with the package. I have done
    the basic learning of python which is way simple compared to the
    stuffs you are talking about. I have just jumped into the extending
    python part of the document. I think I need to master that portion.
    Your code snippet was quite simple and it explained me very well than
    the tutorial. HAts off to u!!

    Bjorn, Marc, Ben, I also noted your comments and I am working on them.
    I have gone through the Python Classes portion and I need to workout
    quite a lot excercises to make myself familiar with Python. I am gonna
    start doing that right away.

    Coming to the complete explanation of what I am upto:

    I am doing my research in Pervasive space environments filled with
    sensors and actuators. I am just trying to modify an existing language
    which suits that kind of environment. Pervasive space environment is
    filled with sensors and actuators and everything depends on their
    state and behavior ie on the sensor readings and actuator states. So
    why not express a language variable in terms of sensors and actuators?
    THis is my idea of working around and trying to introduce new data
    types for these sensor and actuator variables which store sensor
    readings and actuator states.

    I think this explains a bit clearly on what I am looking
    for..Suggestion and ideas are kindly appreciated.

    Thanks in advance and for your time.
    Maximus Decimus, Aug 1, 2007
    #8
  9. Maximus Decimus

    Paul McGuire Guest

    On Aug 1, 3:10 pm, Maximus Decimus <> wrote:
    >
    > Your code snippet was quite simple and it explained me very well than
    > the tutorial. HAts off to u!!
    >

    Thanks!

    <snip>

    >
    > I am doing my research in Pervasive space environments filled with
    > sensors and actuators. I am just trying to modify an existing language
    > which suits that kind of environment. Pervasive space environment is
    > filled with sensors and actuators and everything depends on their
    > state and behavior ie on the sensor readings and actuator states. So
    > why not express a language variable in terms of sensors and actuators?
    > THis is my idea of working around and trying to introduce new data
    > types for these sensor and actuator variables which store sensor
    > readings and actuator states.
    >
    > I think this explains a bit clearly on what I am looking
    > for..Suggestion and ideas are kindly appreciated.
    >
    > Thanks in advance and for your time.


    You got me thinking about a little exercise I did a long time ago,
    using overloaded operators to help model circuits of resistors in
    series and parallel. Here is one way to model some physical objects
    using notation that is particular to their own domain (and even
    counter to the conventional uses of that notation, as noted in the
    docstrings). See below:

    class Resistor(object):
    def __init__(self,r):
    self.resistance = r

    def __sub__(self,other):
    """R1-R2 looks like two resistors in series, nevermind
    that we are using a subtraction operator to perform
    what is essentially an addition operation. Think of
    the horizontal line as a circuit connecting the two
    resistors."""
    return Resistor(self.resistance + other.resistance)

    def __or__(self,other):
    "R1 | R2 looks like two resistors in parallel, sort of"
    return Resistor(float(self.resistance * other.resistance) /
    float(self.resistance + other.resistance))

    def colorCode(self):
    """insert nifty routine here to convert self.resistance to
    sequence of color bands"""
    pass

    def __repr__(self):
    return "Resistor(%s)" % self.resistance

    r1 = Resistor(100)
    r2 = Resistor(200)

    print r1-r2 # resistors in series
    print r1 | r2 # resistors in parallel
    print r1 | r1 | r2 # three resistors in parallel, not so easy math
    print (r1 - r1) | r2 # two resistors in series in parallel with
    another
    print r1 - (r1 | r1)

    Prints:

    Resistor(300)
    Resistor(66.6666666667)
    Resistor(40.0)
    Resistor(100.0)
    Resistor(150.0)

    This is just using vanilla Python with some tricky operator
    interpreting.

    You can also do some syntax manipulation using the import hooks (first
    shown to me by Wilson Fowlie). Here is an example of modeling an in-
    memory state machine, with some customized syntax. (http://
    www.geocities.com/ptmcg/python/stateMachine.html) This is more
    involved, and requires a custom parser, but let's not get distracted
    down that path...

    Just some more alternatives to mucking about in the Python compiler.

    -- Paul
    Paul McGuire, Aug 2, 2007
    #9
  10. Maximus Decimus

    Paul McGuire Guest

    On Aug 1, 8:08 am, Paul McGuire <> wrote:
    >
    > def add(self,other):
    > if len(self.contents) < self.capacity:
    > self.contents.append( other )
    > else:
    > raise ValueError("can't add any more to this Box")
    >


    I guess that really should raise an OverflowError...

    -- Paul
    Paul McGuire, Aug 2, 2007
    #10
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Steve Knight
    Replies:
    2
    Views:
    743
    Steve Knight
    Oct 10, 2003
  2. Torsten Bronger
    Replies:
    3
    Views:
    1,975
    Thomas Heller
    Mar 9, 2006
  3. ramu
    Replies:
    2
    Views:
    314
    rlblaster
    Feb 20, 2006
  4. Gary Roach
    Replies:
    0
    Views:
    103
    Gary Roach
    Sep 1, 2013
  5. Fábio Santos
    Replies:
    0
    Views:
    115
    Fábio Santos
    Sep 4, 2013
Loading...

Share This Page