Execute commands from file

[tmp123]

Hello,

Thanks for your time.

We have very big files with python commands (more or less, 500000
commands each file).

It is possible to execute them command by command, like if the
commands was typed one after the other in a interactive session?

( Better using command flags than with an small script like "while 1:
input()" )

Thanks a lot.

 

[Marc 'BlackJack' Rintsch]

We have very big files with python commands (more or less, 500000
commands each file).

It is possible to execute them command by command, like if the
commands was typed one after the other in a interactive session?

Take a look at the `code` module in the standard library:

In [31]: code?
Type: module
Base Class: <type 'module'>
String Form: <module 'code' from '/usr/lib/python2.4/code.pyc'>
Namespace: Interactive
File: /usr/lib/python2.4/code.py
Docstring:
Utilities needed to emulate Python's interactive interpreter.

Ciao,
Marc 'BlackJack' Rintsch

 

[Steve Holden]

Hello,

Thanks for your time.

We have very big files with python commands (more or less, 500000
commands each file).

Those are BIG programs. Presumably other programs are writing them?

It is possible to execute them command by command, like if the
commands was typed one after the other in a interactive session?

You need to look for "pdb", the interactive Python debugger. This is
capable of single-step operations, and supports breakpoints.

( Better using command flags than with an small script like "while 1:
input()" )

Thanks a lot.

You are pretty much going to have to run pdb then trigger your code by
calling a pdb method with a function in your code as an argument, if I
am remembering correctly how it works.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
------------------ Asciimercial ---------------------
Get on the web: Blog, lens and tag your way to fame!!
holdenweb.blogspot.com squidoo.com/pythonology
tagged items: del.icio.us/steve.holden/python
All these services currently offer free registration!
-------------- Thank You for Reading ----------------

 

[Martin Blume]

We have very big files with python commands
(more or less, 500000 commands each file).

It is possible to execute them command by command,

inp = open(cmd_file)
for line in inp:
exec line

might help. You don't get quite the same feeling as
"like if the commands was typed one after the other
in a interactive session", but perhaps this helps.

Warning: the code above is without any error checks.
You might also run into security problems, the example
above assumes you trust your input.

HTH. YMMV.
Martin

 

[Steve Holden]

>

inp = open(cmd_file)
for line in inp:
exec line

might help. You don't get quite the same feeling as
"like if the commands was typed one after the other
in a interactive session", but perhaps this helps.

Warning: the code above is without any error checks.
You might also run into security problems, the example
above assumes you trust your input.

HTH. YMMV.
Martin

The problem with this approach is that each line executes without any
connection to the environment created by previous lies.

Try it on a file that reads something like

xxx = 42
print xxx

and you will see NameError raised because the assignment hasn't affected
the environment for the print statement.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
------------------ Asciimercial ---------------------
Get on the web: Blog, lens and tag your way to fame!!
holdenweb.blogspot.com squidoo.com/pythonology
tagged items: del.icio.us/steve.holden/python
All these services currently offer free registration!
-------------- Thank You for Reading ----------------

 

[i3dmaster]

The problem with this approach is that each line executes without any
connection to the environment created by previous lies.

Try it on a file that reads something like

xxx = 42
print xxx

and you will see NameError raised because the assignment hasn't affected
the environment for the print statement.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
------------------ Asciimercial ---------------------
Get on the web: Blog, lens and tag your way to fame!!
holdenweb.blogspot.com squidoo.com/pythonology
tagged items: del.icio.us/steve.holden/python
All these services currently offer free registration!
-------------- Thank You for Reading ----------------

cat file:

x = 100
print x

cat file.py:
#!/usr/bin/python2.4

import os.path
import sys

file, ext = os.path.splitext(sys.argv[0])
f = open(file,'rb')
for i in f:
exec i

./file.py

100

Don't see the problem though.

 

[Douglas Woodrow]

f = open(file,'rb')
for i in f:
exec i

Why are you opening the file in binary mode?

 

[Steve Holden]

The problem with this approach is that each line executes without any
connection to the environment created by previous lies.

Try it on a file that reads something like

xxx = 42
print xxx

and you will see NameError raised because the assignment hasn't affected
the environment for the print statement.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
------------------ Asciimercial ---------------------
Get on the web: Blog, lens and tag your way to fame!!
holdenweb.blogspot.com squidoo.com/pythonology
tagged items: del.icio.us/steve.holden/python
All these services currently offer free registration!
-------------- Thank You for Reading ----------------

cat file:

x = 100
print x

cat file.py:
#!/usr/bin/python2.4

import os.path
import sys

file, ext = os.path.splitext(sys.argv[0])
f = open(file,'rb')
for i in f:
exec i

./file.py

100

Don't see the problem though.

No, because there isn't one. Now try adding a function definition and
see how well it works.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
------------------ Asciimercial ---------------------
Get on the web: Blog, lens and tag your way to fame!!
holdenweb.blogspot.com squidoo.com/pythonology
tagged items: del.icio.us/steve.holden/python
All these services currently offer free registration!
-------------- Thank You for Reading ----------------

 

[Maric Michaud]

Steve Holden a écrit :

Martin Blume wrote:
>
We have very big files with python commands
(more or less, 500000 commands each file).
It is possible to execute them command by command,
inp = open(cmd_file)
for line in inp:
exec line
The problem with this approach is that each line executes without any
connection to the environment created by previous lies.

Try it on a file that reads something like

xxx = 42
print xxx

cat file:

x = 100
print x

cat file.py:
#!/usr/bin/python2.4

import os.path
import sys

file, ext = os.path.splitext(sys.argv[0])
f = open(file,'rb')
for i in f:
exec i

./file.py

100

Don't see the problem though.

No, because there isn't one. Now try adding a function definition and
see how well it works.

regards
Steve

This is just a problem with indentation and blocks of code, the
followong will do :

commands = open("commands")
namespace, block = {}, ""
for line in commands :
line=line[:-1]
if not line : continue
if line[0].isspace() :
block += '\n' + line
continue
else :
if block.strip() :
exec block in namespace
block = line

exec block in namespace
print dict((k, v) for k, v in namespace.items() if k != "__builtins__")


with commands containing :

"""

x = 5

def toto(arg) :
print arg

def inner() :
print arg*arg

inner()


toto(x)
"""

output :
5
25
{'x': 5, 'toto': <function toto at 0x01D30C70>}



(sorry Steve for the private mail)

--
_____________

Maric Michaud
_____________

Aristote - www.aristote.info
3 place des tapis
69004 Lyon
Tel: +33 4 26 88 00 97
Mobile: +33 6 32 77 00 21

 

[Martin Blume]

Try it on a file that reads something like

xxx = 42
print xxx

and you will see NameError raised because the assignment
hasn't affected the environment for the print statement.

[...]

No, because there isn't one. Now try adding a function
definition and see how well it works.

C:\temp>more question.py
xxx=42
print xxx
def sowhat():
print xxx

print xxx


C:\temp>c:\programme\python\python
Python 2.4 (#60, Nov 30 2004, 11:49:19)
[MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license"
for more information.

42
42

42


Seems to work great to me.

OTOH, this doesn't:.... exec l
....
42
Traceback (most recent call last):
File "<stdin>", line 2, in ?
File "<string>", line 1
def sowhat():
^
SyntaxError: unexpected EOF while parsing


So it seems to depend on the way the file is read.


Regards
Martin

 

[i3dmaster]

Why are you opening the file in binary mode?

'b' is generally useful on systems that don't treat binary and text
files differently. It will improve portability.

 

[Steve Holden]

Try it on a file that reads something like

xxx = 42
print xxx

and you will see NameError raised because the assignment
hasn't affected the environment for the print statement.

[...]

No, because there isn't one. Now try adding a function
definition and see how well it works.

C:\temp>more question.py
xxx=42
print xxx
def sowhat():
print xxx

print xxx


C:\temp>c:\programme\python\python
Python 2.4 (#60, Nov 30 2004, 11:49:19)
[MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license"
for more information.

42
42

42


Seems to work great to me.

OTOH, this doesn't:... exec l
...
42
Traceback (most recent call last):
File "<stdin>", line 2, in ?
File "<string>", line 1
def sowhat():
^
SyntaxError: unexpected EOF while parsing


So it seems to depend on the way the file is read.

It depends on the way the lines of the file are executed, not how they
are read. And you may remember the original poster was proposing this:

inp = open(cmd_file)

> for line in inp:
> exec line

As for your first example, why not just use execfile() ?

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
------------------ Asciimercial ---------------------
Get on the web: Blog, lens and tag your way to fame!!
holdenweb.blogspot.com squidoo.com/pythonology
tagged items: del.icio.us/steve.holden/python
All these services currently offer free registration!
-------------- Thank You for Reading ----------------

 

[Dennis Lee Bieber]

'b' is generally useful on systems that don't treat binary and text
files differently. It will improve portability.

Pardon... I guess I'm interpreting this differently.

"b" is needed for binary files on systems that /do/ treat binary
differently from text. And it does add to portability only in that it
has no effect on those that treat all files the same.

However, as I recall the thread, the intent is to process text lines
from a file -- and using "b" is going to affect how the line endings are
being treated.
--
Wulfraed Dennis Lee Bieber KD6MOG
(e-mail address removed) (e-mail address removed)
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: (e-mail address removed))
HTTP://www.bestiaria.com/

 

[Douglas Woodrow]

"b" is needed for binary files on systems that /do/ treat binary
differently from text. And it does add to portability only in that it
has no effect on those that treat all files the same.

However, as I recall the thread, the intent is to process text lines
from a file -- and using "b" is going to affect how the line endings are
being treated.

Yes that was my understanding too, Dennis, and the reason I queried it
in the first place. I had to remove the "b" option in order to get the
sample code to work under Windows, because the standard line termination
under Windows is carriage return + linefeed (\r\n).

Of course if I manually edit the command file so that it only has a
linefeed character at the end of each line, the binary mode works.

So I think i3dmaster's method is only portable as long as the command
file is created with unix-style line termination.

 

[Martin Blume]

[ difference between exec open(fname).read()
and for line in open(fname): exec line ]

So it seems to depend on the way the file is read.

It depends on the way the lines of the file are executed,
not how they are read.

Could you elaborate a little bit more on the difference?
I assumed that because read() reads the whole file, the
body of my function sowhat() is present, so that it can
be parsed while the invocation of exec is still running.
If it is read and exec'd line by line, the definition of
the function is still left open at the moment exec() ends,
causing the "EOF" error. Hence my statement, "it depends
on the way the file is read".

And you may remember the original poster was
proposing this:

inp = open(cmd_file)
for line in inp:
exec line

As for your first example, why not just use execfile() ?

I assume that
execfile(fname)
is equivalent to
exec open(fname).read() ?


Regards
Martin

 

[Steve Holden]

[ difference between exec open(fname).read()
and for line in open(fname): exec line ]

So it seems to depend on the way the file is read.

It depends on the way the lines of the file are executed,
not how they are read.

Could you elaborate a little bit more on the difference?
I assumed that because read() reads the whole file, the
body of my function sowhat() is present, so that it can
be parsed while the invocation of exec is still running.
If it is read and exec'd line by line, the definition of
the function is still left open at the moment exec() ends,
causing the "EOF" error. Hence my statement, "it depends
on the way the file is read".

I simply meant that the whole source has to be presented to the exec
statement and not chunked into lines.

Clearly I could read all the source in with

lines = open(cmd_file).readlines()

but if you then proceed to try and execute the source line by line as in

for l in lines:
exec l

you will hit problems because of the disjoint nature of the execution
which will breal up indented suites and so on.

I was probably just a little over-zealous in pursuing correct English
usage, in which case please accept my apology.

I assume that
execfile(fname)
is equivalent to
exec open(fname).read() ?

Pretty much.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
------------------ Asciimercial ---------------------
Get on the web: Blog, lens and tag your way to fame!!
holdenweb.blogspot.com squidoo.com/pythonology
tagged items: del.icio.us/steve.holden/python
All these services currently offer free registration!
-------------- Thank You for Reading ----------------

 

[Martin Blume]

I simply meant that the whole source has to be presented
to the exec statement and not chunked into lines.

That's what I meant: With exec open(f).read() it is not
broken into several exec invocations.

I was probably just a little over-zealous in pursuing
correct English usage, in which case please accept
my apology.

The apology is on my part, I didn't explain my thinking
clearly enough.


Thanks for your explanations. Makes my newbie understanding
of Python much more robust.

Regards
Martin