R BATCH jobs from Python

[Tim Churches]

hello,

i have been attempting to run BATCH jobs on WinXP/Server2003 using
Python. It seems that my questions are similar to those of a previous
OPs. Unfortunately, what seemed like a simple question/request either
digressed or was misinterpreted to the tune of 19 posts:

Author: Moosebumps
Subject (1): Batch commands on Windows
Subject (2): os.system always opens new window on Windows XP/2000

here is the operation that works, but isn't actually carried out in
Python:

Start->Run "cmd" ***command line appears***

then i enter the following commands (i copied and pasted the whole
session so that you can see exactly what i did):
******************************************************
Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

C:\Documents and Settings\BRdir>cd c:\Applications\R\rw1091\bin

C:\Applications\R\rw1091\bin>Rcmd BATCH c:\test.txt

C:\Applications\R\rw1091\bin>
******************************************************

now, the whole point here is to send the following R script to the
program c:\Applications\R\rw1091\bin\Rcmd.exe, i.e. the file cited as
"c:\test.txt" contains the following code:

********************************************************
library(lattice)
x<-read.csv('C:\\Data\\EData\\eFile.txt')
a<-as.matrix(x[,1])
b<-as.matrix(x[,2])
c<-as.matrix(x[,3])
X11()
levelplot(c~a*b)
savePlot(filename='c:\\test2',type='jpeg')
********************************************************

now, the procedure i have just outlined works absolutely perfect. i
have spent some time trying to do this from Python; this includes
looking at posts and references from everywhere i was able to find
them. i still can't run an analogous procedure from python. i have
attempted to use os.system, the popen family. i feel like part of the
issue is that i can't find enough simple examples on how to use these
tools.

Have you considered RPy, which embeds R in Python? See http://rpy.sf.net

Tim C

 

[Benjamin Scott]

hello,

i have been attempting to run BATCH jobs on WinXP/Server2003 using
Python. It seems that my questions are similar to those of a previous
OPs. Unfortunately, what seemed like a simple question/request either
digressed or was misinterpreted to the tune of 19 posts:

Author: Moosebumps
Subject (1): Batch commands on Windows
Subject (2): os.system always opens new window on Windows XP/2000

here is the operation that works, but isn't actually carried out in
Python:

Start->Run "cmd" ***command line appears***

then i enter the following commands (i copied and pasted the whole
session so that you can see exactly what i did):
******************************************************
Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

C:\Documents and Settings\BRdir>cd c:\Applications\R\rw1091\bin

C:\Applications\R\rw1091\bin>Rcmd BATCH c:\test.txt

C:\Applications\R\rw1091\bin>
******************************************************

now, the whole point here is to send the following R script to the
program c:\Applications\R\rw1091\bin\Rcmd.exe, i.e. the file cited as
"c:\test.txt" contains the following code:

********************************************************
library(lattice)
x<-read.csv('C:\\Data\\EData\\eFile.txt')
a<-as.matrix(x[,1])
b<-as.matrix(x[,2])
c<-as.matrix(x[,3])
X11()
levelplot(c~a*b)
savePlot(filename='c:\\test2',type='jpeg')
********************************************************

now, the procedure i have just outlined works absolutely perfect. i
have spent some time trying to do this from Python; this includes
looking at posts and references from everywhere i was able to find
them. i still can't run an analogous procedure from python. i have
attempted to use os.system, the popen family. i feel like part of the
issue is that i can't find enough simple examples on how to use these
tools.

thanks in advance for your feedback,

benjamin scott

 

[Terry Hancock]

Author: Moosebumps
Subject (1): Batch commands on Windows
Subject (2): os.system always opens new window on Windows XP/2000

here is the operation that works, but isn't actually carried out in
Python:

Start->Run "cmd" ***command line appears***

then i enter the following commands (i copied and pasted the whole
session so that you can see exactly what i did):
******************************************************
Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

C:\Documents and Settings\BRdir>cd c:\Applications\R\rw1091\bin

C:\Applications\R\rw1091\bin>Rcmd BATCH c:\test.txt

C:\Applications\R\rw1091\bin>
******************************************************

It would have been better to show us the Python code that *didn't*
work. ;-)

But I suspect it looks like this:

os.system('cd c:\Applications\R\rw1091\bin')
os.system('Rcmd BATCH c:\test.txt')

and your complaint is that the first 'cd' doesn't affect
the second command's context (i.e. they are in separate
shells).

You are right that os.system() uses separate shells, and
this is what it's supposed to do.

There are several fixes for this particular problem. One
is to do the 'cd' in python:

os.chdir('c:\Applications\R\rw1091\bin')
os.system('Rcmd BATCH c:\test.txt')

Another is to just run the command directly:

os.system('c:\Applications\R\rw1091\bin\Rcmd BATCH c:\test.txt')

Either should work fine for your example.

And, of course, it should be mentioned that there exists
a set of R bindings for Python so you could drive R directly,
instead of kludging a script driven from a system call in
Python.

You can also do it with popen, but it's probably less
intuitive than the solutions above (in general, popen isn't
really worth it unless you want to pipe the stdin/stdout
through your program -- which you don't seem to need here?).

Cheers,
Terry

 

[Heiko Wundram]

Am Sonntag, 21. November 2004 01:04 schrieb Terry Hancock:

But I suspect it looks like this:

os.system('cd c:\Applications\R\rw1091\bin')
os.system('Rcmd BATCH c:\test.txt')

Look out for the snake's escaping rules... I bet you meant:

os.system(r'cd c:\Applications\R\rw1091\bin')
os.system(r'Rcmd BATCH c:\test.txt')

So that the backslash doesn't escape the next character, but is verbatim...
That's one reason to hate windows...

Heiko.

 

[Istvan Albert]

them. i still can't run an analogous procedure from python. i have
attempted to use os.system, the popen family. i feel like part of the

This is what I use (works on windows and unix):

inf, outf, errf = os.popen3("R -slave < myscript.R")

You need to have R in your path of course.

Istvan.

 

[Benjamin Scott]

Author: Moosebumps
Subject (1): Batch commands on Windows
Subject (2): os.system always opens new window on Windows XP/2000

here is the operation that works, but isn't actually carried out in
Python:

Start->Run "cmd" ***command line appears***

then i enter the following commands (i copied and pasted the whole
session so that you can see exactly what i did):
******************************************************
Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

C:\Documents and Settings\BRdir>cd c:\Applications\R\rw1091\bin

C:\Applications\R\rw1091\bin>Rcmd BATCH c:\test.txt

C:\Applications\R\rw1091\bin>
******************************************************

It would have been better to show us the Python code that *didn't*
work. ;-)

i was trying to spare you the pain...

But I suspect it looks like this:

os.system('cd c:\Applications\R\rw1091\bin')
os.system('Rcmd BATCH c:\test.txt')

yes, in fact it did look something like this.

and your complaint is that the first 'cd' doesn't affect
the second command's context (i.e. they are in separate
shells).

yes, i couldn't pass multiple commands. that was the main problem.
the shell would immediately close once it was opened.

You are right that os.system() uses separate shells, and
this is what it's supposed to do.

There are several fixes for this particular problem. One
is to do the 'cd' in python:

os.chdir('c:\Applications\R\rw1091\bin')
os.system('Rcmd BATCH c:\test.txt')

this worked... i think i changed the backslashes to double
backslashes, but with minor alterations it worked. thanks very much.

Another is to just run the command directly:

os.system('c:\Applications\R\rw1091\bin\Rcmd BATCH c:\test.txt')

Either should work fine for your example.

And, of course, it should be mentioned that there exists
a set of R bindings for Python so you could drive R directly,
instead of kludging a script driven from a system call in
Python.

yes, the following poster, Tim C, mentions RPy. i have tinkered with
RPy and with RSPython from omegahat. RPy seems to work as you would
want/expect. just takes a bit of time to get used to the syntax. i
think this might be overkill in this case unless there are major
performance differences of which i am unaware.

You can also do it with popen, but it's probably less
intuitive than the solutions above (in general, popen isn't
really worth it unless you want to pipe the stdin/stdout
through your program -- which you don't seem to need here?).

this is the type of stuff that adds unnecessary confusion for
developers attempting to use new tools. that is, there seems to be
multiple tools that all have overlapping functionality, but they are
administered differently. this was part of the motivation for writing
popen5 (renamed: subprocess), but in order for this tool to be
accessible to new developers it needs more examples in its
documentation... or there needs to be a crossreference with more
examples existing elsewhere. popen5 has many examples on how the new
tool is related to the handful of tools that came before it, but this
isn't particularly helpful to a newcomer to the tool.

tools that allow interprocess/application communication seem like they
add so much value to the parent application; i am surprised i wasn't
able to find some kind of beginner's "Quick Start" online somewhere.
anyhow, that's my $.02.

Cheers,
Terry

thanks again for your feedback,

benjamin scott