[ANN] DoIt 0.1.0 Released (build tool)

Discussion in 'Python' started by Eduardo Schettino, Apr 19, 2008.

  1. DoIt - A task execution tool (build-tool)
    =========================================

    This is the first public release of DoIt

    Website: http://python-doit.sourceforge.net/
    Release: DoIt 0.1.0
    License: MIT

    About
    -----

    DoIt is a build tool that focus not only on making/building things but on
    executing any kind of tasks in an efficient way. Designed to be easy to use
    and "get out of your way".

    DoIt like most build tools is used to execute tasks defined in a
    configuration file. Configuration files are python modules. The tasks can be
    python functions (or any callable) or an external shell script. DoIt
    automatically keeps track of declared dependencies executing only tasks that
    needs to be update (based on which dependencies have changed).

    In DoIt, unlike most(all?) build-tools, a task doesn't need to define a target
    file to use the execute only if not up-to-date feature. This make DoIt
    specially suitable for running test suites.

    DoIt can be used to perform any task or build anything, though it doesn't
    support automatic dependency discovery for any language.

    Cheers,
    Eduardo
    Eduardo Schettino, Apr 19, 2008
    #1
    1. Advertising

  2. Eduardo Schettino

    John Machin Guest

    Eduardo Schettino wrote:
    > DoIt - A task execution tool (build-tool)
    > =========================================
    >
    > This is the first public release of DoIt
    >
    > Website: http://python-doit.sourceforge.net/
    > Release: DoIt 0.1.0
    > License: MIT
    >
    > About
    > -----
    >
    > DoIt is a build tool that focus not only on making/building things but on
    > executing any kind of tasks in an efficient way. Designed to be easy to use
    > and "get out of your way".


    You may like to consider the possibility of confusion caused by the
    similarity of some characters in some fonts (DoIt, Do1t, Dolt) ...
    google("dictionary dolt") :)
    John Machin, Apr 19, 2008
    #2
    1. Advertising

  3. On Sun, Apr 20, 2008 at 2:04 AM, John Machin <> wrote:
    > You may like to consider the possibility of confusion caused by the
    > similarity of some characters in some fonts (DoIt, Do1t, Dolt) ...
    > google("dictionary dolt") :)
    > --
    > http://mail.python.org/mailman/listinfo/python-list
    >


    hehehehe. i feel like a DOLT

    I never realized that it is confusing. I also didnt know this word
    "dolt" before.
    i wont use capital letters anymore.
    thanks
    Eduardo Schettino, Apr 19, 2008
    #3
  4. Python "make" like tools (was Re: [ANN] DoIt 0.1.0 Released (buildtool))

    Eduardo Schettino wrote:

    > DoIt is a build tool that focus not only on making/building things but on
    > executing any kind of tasks in an efficient way. Designed to be easy to use
    > and "get out of your way".


    I took a look at dolt syntax, and saw this:

    QQQ

    def create_folder(path):
    """Create folder given by "path" if it doesnt exist"""
    if not os.path.exists(path):
    os.mkdir(path)
    return True

    def task_create_build_folder():
    buildFolder = jsPath + "build"
    return {'action':create_folder,
    'args': (buildFolder,)
    }

    QQQ

    Wouldn't it be more convenient to provide syntax like this:

    @task("create_build_folder")
    @depend("dep1 some_other_dep")
    def buildf():
    buildFolder = jsPath + "build"
    create_folder(buildFolder)

    I find the doit syntax a bit cumbersome, especially as you can avoid
    'args' by just returning a lamda in 'action'.

    I've looked around a bit for python "make" replacement, but there does
    not seem to be a simple & straightforward solution around (read -
    straight-python syntax, one .py file installation, friendly license).
    Ville M. Vainio, Apr 21, 2008
    #4
  5. Eduardo Schettino

    Paul Boddie Guest

    Re: Python "make" like tools (was Re: [ANN] DoIt 0.1.0 Released(build tool))

    On 21 Apr, 16:51, "Ville M. Vainio" <> wrote:
    >
    > Wouldn't it be more convenient to provide syntax like this:
    >
    > @task("create_build_folder")
    > @depend("dep1 some_other_dep")
    > def buildf():
    > buildFolder = jsPath + "build"
    > create_folder(buildFolder)


    I'd want to make the "grunt work" a bit easier before breaking out the
    decorators.

    > I find the doit syntax a bit cumbersome, especially as you can avoid
    > 'args' by just returning a lamda in 'action'.
    >
    > I've looked around a bit for python "make" replacement, but there does
    > not seem to be a simple & straightforward solution around (read -
    > straight-python syntax, one .py file installation, friendly license).


    Have you surveyed the landscape...?

    http://wiki.python.org/moin/ConfigurationAndBuildTools

    I'm inclined to think that Waf would probably meet your requirements:

    http://code.google.com/p/waf/

    Paul
    Paul Boddie, Apr 21, 2008
    #5
  6. Re: Python "make" like tools (was Re: [ANN] DoIt 0.1.0 Released(build tool))

    >
    > I took a look at dolt syntax, and saw this:
    >
    > QQQ
    >
    > def create_folder(path):
    > """Create folder given by "path" if it doesnt exist"""
    > if not os.path.exists(path):
    > os.mkdir(path)
    > return True
    >
    > def task_create_build_folder():
    > buildFolder = jsPath + "build"
    > return {'action':create_folder,
    > 'args': (buildFolder,)
    > }
    >
    > QQQ
    >
    > Wouldn't it be more convenient to provide syntax like this:
    >
    > @task("create_build_folder")
    > @depend("dep1 some_other_dep")
    > def buildf():
    > buildFolder = jsPath + "build"
    > create_folder(buildFolder)
    >
    > I find the doit syntax a bit cumbersome, especially as you can avoid
    > 'args' by just returning a lamda in 'action'.



    My idea was to: do *not* add any new syntax (to avoid being
    cumbersome). It is just python, you dont have to import or subclass
    anything. You just need to create a function that returns a dictionary
    with some predefined keys.

    I though about using decorators in the beginning... but returning a
    dictionary looked easier to implement and more flexible. one important
    feature is how easy to define a group of task with the same action.
    Take a look at the example below on running pychecker in all python
    files from a folder. I couldnt figure out an easy way of doing it with
    decorators.

    import glob;
    pyFiles = glob.glob('*.py')

    def task_checker():
    for f in pyFiles:
    yield {'action': "pychecker %s"% f,
    'name':f,
    'dependencies':(f,)}

    Another advantage of using just a dictionary to define a task is that
    it will be easy to read tasks from a text file (if it is very simple
    and you dont need to write any python script). but not implemented
    yet.

    I though about using decorator for simple python-tasks but in a different way:

    @task
    def create_folder(path):
    """Create folder given by "path" if it doesnt exist"""
    if not os.path.exists(path):
    os.mkdir(path)
    return True

    so if your python function is a task and you will use it only once you
    dont need to define a function for the 'action' and another function
    to create the task. but not implement yet also.

    >
    > I've looked around a bit for python "make" replacement, but there does
    > not seem to be a simple & straightforward solution around (read -
    > straight-python syntax, one .py file installation, friendly license).


    apart from one .py file installation (easy_install is not enough?)
    thats what i am trying to do.

    thanks for the feedback.
    cheers,
    Eduardo
    Eduardo Schettino, Apr 21, 2008
    #6
  7. Re: Python "make" like tools (was Re: [ANN] DoIt 0.1.0 Released(build tool))

    I guess I should post a link to the project in this thread...

    http://python-doit.sourceforge.net/
    Eduardo Schettino, Apr 21, 2008
    #7
  8. Re: Python "make" like tools (was Re: [ANN] DoIt 0.1.0 Released (buildtool))

    Eduardo Schettino wrote:

    >> I find the doit syntax a bit cumbersome, especially as you can avoid
    >> 'args' by just returning a lamda in 'action'.

    >
    >
    > My idea was to: do *not* add any new syntax (to avoid being
    > cumbersome). It is just python, you dont have to import or subclass


    Yeah, decorators get around this.

    > I though about using decorators in the beginning... but returning a
    > dictionary looked easier to implement and more flexible. one important
    > feature is how easy to define a group of task with the same action.
    > Take a look at the example below on running pychecker in all python
    > files from a folder. I couldnt figure out an easy way of doing it with
    > decorators.
    >
    > import glob;
    > pyFiles = glob.glob('*.py')
    >
    > def task_checker():
    > for f in pyFiles:
    > yield {'action': "pychecker %s"% f,
    > 'name':f,
    > 'dependencies':(f,)}


    Perhaps you could do:

    for f in pyFiles:
    @task("checker")
    @depend(f)
    def check():
    c("pychecker %s" % f)

    Never underestimate the magic that is nested scopes and name-agnostic
    function object creation...


    > Another advantage of using just a dictionary to define a task is that
    > it will be easy to read tasks from a text file (if it is very simple
    > and you dont need to write any python script). but not implemented
    > yet.


    It is easy with the above syntax as well.


    > I though about using decorator for simple python-tasks but in a different way:
    >
    > @task
    > def create_folder(path):
    > """Create folder given by "path" if it doesnt exist"""
    > if not os.path.exists(path):
    > os.mkdir(path)
    > return True
    >
    > so if your python function is a task and you will use it only once you
    > dont need to define a function for the 'action' and another function
    > to create the task. but not implement yet also.


    Yeah, this is what I consider much friendlier syntax (the aim is to not
    be much more verbose than make).

    > apart from one .py file installation (easy_install is not enough?)
    > thats what i am trying to do.


    easy_install is not really enough - it introduces a dependency that you
    can't get around by just shipping a short .py file with your project.

    This problem domain seems simple enough that it should be covered by a
    very short and simple module (Paul, the Waf you suggested was ~ 300k, no
    doubt caused by all the c compiler stuff that I don't need).

    'Paver' seems to have the right idea:

    http://www.blueskyonmars.com/projects/paver/index.html

    But it's still *slightly* too big:

    http://bazaar.launchpad.net/~dangoor/paver/main/files
    Ville M. Vainio, Apr 23, 2008
    #8
  9. Re: Python "make" like tools (was Re: [ANN] DoIt 0.1.0 Released(build tool))

    On Wed, Apr 23, 2008 at 8:39 PM, Ville M. Vainio <> wrote:
    >
    > Yeah, decorators get around this.
    >
    >
    > Perhaps you could do:
    >
    > for f in pyFiles:
    > @task("checker")
    > @depend(f)
    > def check():
    > c("pychecker %s" % f)
    >
    > Never underestimate the magic that is nested scopes and name-agnostic
    > function object creation...
    >


    ok. it is possible to do everything with decorators... i agree.
    but i dont want "name-agnostic function object creation". i want to be
    able to execute every single task without executing all other tasks.
    thats why when defining sub-tasks it is required an extra parameter
    "name". I know you could add another decorator for this also :)
    I mean decorators could do the job i dont say they cant. *I* prefer to
    work with dictionaries though. Even if they are a bit more verbose.


    > >
    > > I though about using decorator for simple python-tasks but in a different

    > way:
    > >
    > > @task
    > >
    > > def create_folder(path):
    > > """Create folder given by "path" if it doesnt exist"""
    > > if not os.path.exists(path):
    > > os.mkdir(path)
    > > return True
    > >
    > >
    > > so if your python function is a task and you will use it only once you
    > > dont need to define a function for the 'action' and another function
    > > to create the task. but not implement yet also.
    > >

    >
    > Yeah, this is what I consider much friendlier syntax (the aim is to not be
    > much more verbose than make).
    >
    >


    Maybe it is just a bad example. I tried to put all features on single
    example. but in *real* life I could do like this.

    import os
    jsPath = "./"
    jsFiles = ["file1.js", "file2.js"]
    sourceFiles = [jsPath + f for f in jsFiles]
    compressedFiles = [jsPath + "build/" + f + ".compressed" for f in jsFiles]

    def task_shrink_js():
    # create output folder
    if not os.path.exists(path):
    os.mkdir(path)
    for jsFile,compFile in zip(sourceFiles,compressedFiles):
    action = 'java -jar custom_rhino.jar -c %s > %s'% (jsFile, compFile)
    yield {'action':action,
    'name':jsFile,
    'dependencies':(jsFile,),
    'targets':(compFile,)
    }

    Can I challenge you to do this with any other build tool?

    i guess anything more complicated than this you want to create a
    python function separate from the task definition anyway.

    My examples are from scratch. I dont use a "standard library".

    I could also document the short-cuts :p. I was afraid that including
    the short-cuts would make it more complicated for who is trying to
    learn it.

    if you dont define any dependencies or args to the function you dont
    need to return a dictionary. just the action:

    def task_simple_shell():
    return "echo spam"

    def task_simple_python():
    def say_spam():
    print "spam"
    return True
    return say_spam

    # or from our discussion

    def task_create_build_folder():
    def create_folder():
    path = jsPath + "build"
    if not os.path.exists(path):
    os.mkdir(path)
    return True
    return create_folder


    >
    > > apart from one .py file installation (easy_install is not enough?)
    > > thats what i am trying to do.
    > >

    >
    > easy_install is not really enough - it introduces a dependency that you
    > can't get around by just shipping a short .py file with your project.
    >

    what about shipping a single egg file with the whole package? ( I am
    not very much familiar with this subject)

    > 'Paver' seems to have the right idea:
    >
    > http://www.blueskyonmars.com/projects/paver/index.html
    >


    Thanks for the link. I took a quick look on its documentation.
    It seems that "paver" doesnt keep track of file-dependencies at all.
    So it is more like a setuptools extension commands than a
    "full-feature" build system with "smart" re-build and dependency
    support.

    "doit" target is not on creating releases for python projects only. it
    can do it also, but it can do much more.

    You can read about my motivation to start another build tool project
    on http://schettino72.wordpress.com/2008/04/14/doit-a-build-tool-tale/


    > But it's still *slightly* too big:
    >


    man, it is hard to make you happy :)

    the doit egg file containg the whole packge is 27782 bytes on my
    system. but you also need the command line script 154 bytes.

    cheers,
    Eduardo
    Eduardo Schettino, Apr 23, 2008
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Eduardo Schettino

    ANN: doit 0.2.0 released

    Eduardo Schettino, Apr 17, 2009, in forum: Python
    Replies:
    0
    Views:
    249
    Eduardo Schettino
    Apr 17, 2009
  2. schettino72
    Replies:
    0
    Views:
    300
    schettino72
    Sep 1, 2009
  3. Eduardo Schettino

    [ANN] doit 0.5

    Eduardo Schettino, Dec 2, 2009, in forum: Python
    Replies:
    0
    Views:
    232
    Eduardo Schettino
    Dec 2, 2009
  4. Eduardo Schettino

    [ANN] doit - automation tool 0.8 released

    Eduardo Schettino, May 17, 2010, in forum: Python
    Replies:
    0
    Views:
    236
    Eduardo Schettino
    May 17, 2010
  5. bob
    Replies:
    3
    Views:
    179
Loading...

Share This Page