script files with python (instead of tcsh/bash)?

Discussion in 'Python' started by Esmail, Mar 21, 2009.

  1. Esmail

    Esmail Guest

    Hello all,

    I am wondering if anyone is using python to write script files?

    Right now I have a bigg'ish bash/tcsh script that contain some grep/awk
    command plus various files are processed and created, renamed and
    moved to specific directories. I also write out some gnuplot scripts
    that later get executed to generate .jpg images.

    In any case, the scripts are starting to look pretty hairy and I was
    wondering if it would make sense to re-write them in Python. I am not
    sure how suitable it would be for this.

    I've looked around the web w/o much luck for some examples but come
    short. Any comments/suggestions?

    Thanks,
    Esmail
    Esmail, Mar 21, 2009
    #1
    1. Advertising

  2. Esmail

    Aahz Guest

    In article <>,
    Esmail <> wrote:
    >
    >I am wondering if anyone is using python to write script files?


    These days, I always convert any even slightly complicated script to
    Python.

    >I've looked around the web w/o much luck for some examples but come
    >short. Any comments/suggestions?


    Not sure what you're looking for here -- many things you'd run an
    external program for in scripting can be accomplished with Python library
    calls, and for the rest, you can use the subprocess module (or os.system
    if you have no acces to Python 2.4 or higher).
    --
    Aahz () <*> http://www.pythoncraft.com/

    "Programming language design is not a rational science. Most reasoning
    about it is at best rationalization of gut feelings, and at worst plain
    wrong." --GvR, python-ideas, 2009-3-1
    Aahz, Mar 21, 2009
    #2
    1. Advertising

  3. Esmail

    Esmail Guest

    Aahz wrote:
    > In article <>,
    > Esmail <> wrote:
    >> I am wondering if anyone is using python to write script files?

    >
    > These days, I always convert any even slightly complicated script to
    > Python.


    well .. that sounds encouraging ...

    >> I've looked around the web w/o much luck for some examples but come
    >> short. Any comments/suggestions?

    >
    > Not sure what you're looking for here -- many things you'd run an
    > external program for in scripting can be accomplished with Python library
    > calls, and for the rest, you can use the subprocess module (or os.system
    > if you have no acces to Python 2.4 or higher).


    I have access to 2.5 or more recent. I guess I was looking for some
    example scripts in Python and perhaps the equivalent in bash/tsch to
    show some of the equivalences. I am being impatient, I guess I need to
    dig into the language/library documentation a bit more on my own.

    Esmail
    Esmail, Mar 21, 2009
    #3
  4. Esmail

    Aahz Guest

    In article <>,
    Esmail <> wrote:
    >Aahz wrote:
    >> In article <>,
    >> Esmail <> wrote:
    >>>
    >>> I've looked around the web w/o much luck for some examples but come
    >>> short. Any comments/suggestions?

    >>
    >> Not sure what you're looking for here -- many things you'd run an
    >> external program for in scripting can be accomplished with Python library
    >> calls, and for the rest, you can use the subprocess module (or os.system
    >> if you have no acces to Python 2.4 or higher).

    >
    >I have access to 2.5 or more recent. I guess I was looking for some
    >example scripts in Python and perhaps the equivalent in bash/tsch to
    >show some of the equivalences. I am being impatient, I guess I need to
    >dig into the language/library documentation a bit more on my own.


    If you post a sample script you're trying to convert, you may get some
    responses that show how different people would write it in Python.
    --
    Aahz () <*> http://www.pythoncraft.com/

    "Programming language design is not a rational science. Most reasoning
    about it is at best rationalization of gut feelings, and at worst plain
    wrong." --GvR, python-ideas, 2009-3-1
    Aahz, Mar 21, 2009
    #4
  5. On Sat, 21 Mar 2009 09:26:02 -0400, Esmail <> wrote:
    >
    > I am wondering if anyone is using python to write script files?


    If it can be done in a few simple lines of shell script,
    fine: make it a shell script. But if it's more complex than
    that, Python is clearer. Just my two cents.

    --
    To email me, substitute nowhere->spamcop, invalid->net.
    Peter Pearson, Mar 21, 2009
    #5
  6. Esmail

    Esmail Guest

    Aahz wrote:
    >
    > If you post a sample script you're trying to convert, you may get some
    > responses that show how different people would write it in Python.


    That's a nice suggestion .. I may end up doing this after I do some
    readings, just wanted to make sure this is not too outlandish of an
    idea :)

    Esmail
    Esmail, Mar 21, 2009
    #6
  7. Esmail

    Esmail Guest

    Peter Pearson wrote:
    > On Sat, 21 Mar 2009 09:26:02 -0400, Esmail <> wrote:
    >> I am wondering if anyone is using python to write script files?

    >
    > If it can be done in a few simple lines of shell script,
    > fine: make it a shell script. But if it's more complex than
    > that, Python is clearer. Just my two cents.


    Hi Peter,

    Yes, I agree .. the complexity of the script(s) have evolved to the
    stage that I was thinking moving to Python makes sense (if that's something
    that Python is suitable for - which it appears it is :)

    Cheers,
    Esmail
    Esmail, Mar 21, 2009
    #7
  8. Esmail

    Ned Deily Guest

    In article <gq2rjc$ji$>, Esmail <>
    wrote:
    > Aahz wrote:
    > > In article <>,
    > > Esmail <> wrote:
    > >> I am wondering if anyone is using python to write script files?

    > >
    > > These days, I always convert any even slightly complicated script to
    > > Python.

    > well .. that sounds encouraging ...>
    > >> I've looked around the web w/o much luck for some examples but come
    > >> short. Any comments/suggestions?

    > > Not sure what you're looking for here -- many things you'd run an
    > > external program for in scripting can be accomplished with Python library
    > > calls, and for the rest, you can use the subprocess module (or os.system
    > > if you have no acces to Python 2.4 or higher).

    > I have access to 2.5 or more recent. I guess I was looking for some
    > example scripts in Python and perhaps the equivalent in bash/tsch to
    > show some of the equivalences. I am being impatient, I guess I need to
    > dig into the language/library documentation a bit more on my own.


    Perhaps the recipe for Pyline might give you some ideas on how to write
    python scripts that play well with other scripts.

    <http://code.activestate.com/recipes/437932/>

    --
    Ned Deily,
    Ned Deily, Mar 21, 2009
    #8
  9. Esmail

    Esmail Guest

    Ned Deily wrote:
    >
    > Perhaps the recipe for Pyline might give you some ideas on how to write
    > python scripts that play well with other scripts.
    >
    > <http://code.activestate.com/recipes/437932/>


    ah .. very nice .. thanks!
    Esmail, Mar 22, 2009
    #9
  10. Esmail

    Esmail Guest

    andrew cooke wrote:
    > Esmail wrote:
    >
    > just a quick data point here -


    <..>

    > so you might be better spending the time improving your bash skills than
    > doing what will be largely drudge work in a language you already know.


    I'll have to think about it .. at this point I know both languages about
    equally "well" .. (not really), but Python has more universal applicability,
    so I thought this might be a good reason to hone my Python skills and come
    up with something useful at the same time.

    Thanks for the post, the more data points,the better.

    Cheers,
    Esmail
    Esmail, Mar 22, 2009
    #10
  11. Esmail

    Esmail Guest

    andrew cooke wrote:
    > Esmail wrote:
    >
    > just a quick data point here -


    <..>

    > so you might be better spending the time improving your bash skills than
    > doing what will be largely drudge work in a language you already know.


    I'll have to think about it .. at this point I know both languages about
    equally "well" .. (not really), but Python has more universal applicability,
    so I thought this might be a good reason to hone my Python skills and come
    up with something useful at the same time.

    Thanks for the post, the more data points,the better.

    Cheers,
    Esmail
    Esmail, Mar 22, 2009
    #11
  12. Esmail

    Esmail Guest

    Nick Craig-Wood wrote:
    > Esmail <> wrote:
    >> I am wondering if anyone is using python to write script files?

    >
    > Yes!
    >

    <..>
    >
    > Almost any script that contains a loop I convert into python.
    >
    >> In any case, the scripts are starting to look pretty hairy and I was
    >> wondering if it would make sense to re-write them in Python. I am not
    >> sure how suitable it would be for this.

    >
    > With python you get the advantages of a language with a very clear
    > philosophy which is exceptionally easy to read and write.
    >
    > Add a few classes and unit tests to your scripts and they will be
    > better than you can ever achieve with bash (IMHO).



    Hi Nick,

    thanks for including the script, that really helps. Nice way of
    finding files.

    Two quick questions:

    As a replacement for grep I would use the re module and its methods?

    What about awk which I regularly use to extract fields based on position
    but not column number, what should I be using in Python to do the same?

    The other things I need to do consist of moving files, manipulating file
    names and piping outputs of one command to the next, so I'm digging into
    the documentation as much as I can.

    So much to learn, so little time (but so much fun!)

    Esmail
    Esmail, Mar 22, 2009
    #12
  13. Esmail

    MRAB Guest

    Esmail wrote:
    > Nick Craig-Wood wrote:
    >> Esmail <> wrote:
    >>> I am wondering if anyone is using python to write script files?

    >>
    >> Yes!
    >>

    > <..>
    >>
    >> Almost any script that contains a loop I convert into python.
    >>
    >>> In any case, the scripts are starting to look pretty hairy and I was
    >>> wondering if it would make sense to re-write them in Python. I am not
    >>> sure how suitable it would be for this.

    >>
    >> With python you get the advantages of a language with a very clear
    >> philosophy which is exceptionally easy to read and write.
    >>
    >> Add a few classes and unit tests to your scripts and they will be
    >> better than you can ever achieve with bash (IMHO).

    >
    >
    > Hi Nick,
    >
    > thanks for including the script, that really helps. Nice way of
    > finding files.
    >
    > Two quick questions:
    >
    > As a replacement for grep I would use the re module and its methods?
    >
    > What about awk which I regularly use to extract fields based on position
    > but not column number, what should I be using in Python to do the same?
    >

    Just use string slicing.

    > The other things I need to do consist of moving files, manipulating file
    > names and piping outputs of one command to the next, so I'm digging into
    > the documentation as much as I can.
    >

    The 'os' and 'shutil' modules.

    > So much to learn, so little time (but so much fun!)
    >
    MRAB, Mar 22, 2009
    #13
  14. En Sun, 22 Mar 2009 11:05:22 -0300, MRAB <>
    escribió:
    > Esmail wrote:
    >> Nick Craig-Wood wrote:
    >>> Esmail <> wrote:
    >>>> I am wondering if anyone is using python to write script files?


    >> Two quick questions:
    >> As a replacement for grep I would use the re module and its methods?


    Perhaps; but strings have methods too (`"abc" in line` is easier to read,
    and faster, than the corresponding r.e.)

    >> The other things I need to do consist of moving files, manipulating file
    >> names and piping outputs of one command to the next, so I'm digging into
    >> the documentation as much as I can.
    >>

    > The 'os' and 'shutil' modules.


    And for executing external commands with piping, I'd add the subprocess
    module.

    --
    Gabriel Genellina
    Gabriel Genellina, Mar 22, 2009
    #14
  15. Esmail

    Esmail Guest

    MRAB wrote:
    >
    >> Two quick questions:
    >>
    >> As a replacement for grep I would use the re module and its methods?
    >>
    >> What about awk which I regularly use to extract fields based on position
    >> but not column number, what should I be using in Python to do the same?
    >>

    > Just use string slicing.


    Would that be equivalent to splitting the string and then subscripting
    on the result?

    That seems to be the closest to awk .. wouldn't string slicing depend on
    column numbers for the line (so to speak)i.e, specific know index values?

    >> The other things I need to do consist of moving files, manipulating file
    >> names and piping outputs of one command to the next, so I'm digging into
    >> the documentation as much as I can.
    >>

    > The 'os' and 'shutil' modules.


    Excellent, thanks for the pointers. I was aware of the os module, but
    didn't know about the shutil module. The more I dig into Python, the
    more I like it and the more I'm impressed. (I do miss bock comments,
    when you are trying out new things that is useful .. I know I can use
    """ as a substitute though).

    Thanks again,
    Esmail
    Esmail, Mar 23, 2009
    #15
  16. Esmail

    Esmail Guest

    Hi Gabriel,

    Gabriel Genellina wrote:
    > En Sun, 22 Mar 2009 11:05:22 -0300, MRAB <>
    > escribió:
    >> Esmail wrote:
    >>> Nick Craig-Wood wrote:
    >>>> Esmail <> wrote:

    <..>
    >>> As a replacement for grep I would use the re module and its methods?

    >
    > Perhaps; but strings have methods too (`"abc" in line` is easier to
    > read, and faster, than the corresponding r.e.)


    I'm a big fan of readability (which is one reason I am attracted
    to Python) .. thanks for pointing out the alternative, good to have
    options.

    >>> The other things I need to do consist of moving files, manipulating file
    >>> names and piping outputs of one command to the next, so I'm digging into
    >>> the documentation as much as I can.
    >>>

    >> The 'os' and 'shutil' modules.

    >
    > And for executing external commands with piping, I'd add the subprocess
    > module.


    I will take a look at the subprocess module, thanks!

    Esmail
    Esmail, Mar 23, 2009
    #16
  17. Esmail

    Esmail Guest

    Hello again Nick,

    thanks for the additional script example. I was able to put
    something together where I read the whole file into a list
    as a series of lines (via readlines()) and then loop through
    the lines seeing if the target string was "in" the line .. seems
    to have worked reasonably well.

    I am sure over time I will pick up the more Python(ic?) ways of
    doing things.


    > I presume you mean something like this
    >
    > ... | awk '{print $2}'
    >
    > In python, assuming you've got the line in the "line" variable, then
    >
    > In python an equivalent of the above would be
    >
    > import fileinput
    > for line in fileinput.input():
    > print line.split()[1]



    cool .. that is what I came up with too!

    > Note that the fileinput module is really useful for making shell
    > command replacements!


    yes, I am going to have to process a number of files, so I'm also
    looking at glob and os.walk(??)

    >> The other things I need to do consist of moving files, manipulating file
    >> names and piping outputs of one command to the next, so I'm digging into
    >> the documentation as much as I can.

    >
    > Read up on the os module and the subprocess module. You'll find you
    > need to do much less piping with python as with shell because it has
    > almost everything you'll need built in.
    >
    > Using built in functions is much quicker than fork()-ing an external
    > command too.


    Thanks again for all the useful info and examples, this really has been
    a great help.

    Esmail
    Esmail, Mar 23, 2009
    #17
  18. Esmail

    Alan G Isaac Guest

    On 3/21/2009 9:26 AM Esmail apparently wrote:
    > I also write out some gnuplot scripts
    > that later get executed to generate .jpg images.



    See Gnuplot.py

    Alan Isaac
    Alan G Isaac, Mar 23, 2009
    #18
  19. Esmail

    Esmail Guest

    Alan G Isaac wrote:
    > On 3/21/2009 9:26 AM Esmail apparently wrote:
    >> I also write out some gnuplot scripts
    >> that later get executed to generate .jpg images.

    >
    >
    > See Gnuplot.py


    Thanks Alan, I will!

    Esmail
    Esmail, Mar 23, 2009
    #19
  20. Esmail <> wrote:
    > Hello again Nick,
    >
    > thanks for the additional script example. I was able to put
    > something together where I read the whole file into a list
    > as a series of lines (via readlines()) and then loop through
    > the lines seeing if the target string was "in" the line .. seems
    > to have worked reasonably well.
    >
    > I am sure over time I will pick up the more Python(ic?) ways of
    > doing things.


    Here's a more Pythonic way to do that:

    with open('somefile') as f:
    for line in f:
    if 'somestring' in line:
    #do something

    In other words, you don't have to read the lines into a list first if all
    you are going to do is iterate through them. (The 'with' clause closes
    the file at block exit...which is overkill if this is all the program
    is doing since the file will be closed at program termination anyway,
    but is a good habit to get in to.)

    --
    R. David Murray http://www.bitdance.com
    R. David Murray, Mar 24, 2009
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. anuttka

    tcsh and java: script stuck

    anuttka, Jan 6, 2004, in forum: Java
    Replies:
    5
    Views:
    727
    anuttka
    Jan 12, 2004
  2. William Park

    (patch for Bash) GTK+2 + Bash

    William Park, Jul 11, 2005, in forum: XML
    Replies:
    3
    Views:
    2,293
    William Park
    Jul 14, 2005
  3. David Durkee
    Replies:
    3
    Views:
    537
    Dan Sommers
    Jul 22, 2005
  4. Gali
    Replies:
    1
    Views:
    515
    Matthias Buelow
    Jul 30, 2007
  5. Esmail
    Replies:
    0
    Views:
    379
    Esmail
    Mar 21, 2009
Loading...

Share This Page