How to program in Python to run system commands in 1000s of servers

B

Babu

Here is my problem: Want to program in python to run sysadmin
commands across 1000s of servers and gather the result in one place.
Many times the commands need to be run as root. We cannot use ssh as
root remote connectivity as well. What are the different ways of
programming in python to achieve this?
 
G

geremy condra

Here is my problem:  Want to program in python to run sysadmin
commands across 1000s of servers and gather the result in one place.
Many times the commands need to be run as root.  We cannot use ssh as
root remote connectivity as well.  What are the different ways of
programming in python to achieve this?

There are a bajillion ways to do it badly, but SSH sounds like the
right tool for the job here. You really don't want your remote admin
system compromised, and fabric makes this kind of thing really much
less painful.

Geremy Condra
 
R

Roy Smith

geremy condra said:
There are a bajillion ways to do it badly, but SSH sounds like the
right tool for the job here. You really don't want your remote admin
system compromised, and fabric makes this kind of thing really much
less painful.

Agreed on the fabric (fabfile.org) recommendation. We've been using it
for about 6 months. Very handy.

I'm not sure how to parse:
We cannot use ssh as root remote connectivity as well.

but with 1000's of servers, I really don't see any alternative to ssh,
with key authentication. You don't really propose to type passwords at
1000's of machines, do you?

As far as fabric goes, it's not perfect, but it's pretty good and if you
try to roll your own alternative, you will likely 1) waste a lot of time
and money and 2) end up with an inferior solution.
 
A

Anssi Saari

Roy Smith said:
I'm not sure how to parse:

but with 1000's of servers, I really don't see any alternative to ssh,
with key authentication. You don't really propose to type passwords at
1000's of machines, do you?

I guess it might mean someone decided to config sshd with
PermitRootLogin no... I believe this is common? I don't think it's a
particularly good idea, especially for a large scale deployment.

So I guess there may be some config needed for the machines before
they can be remotely administrated in an automatic fashion.
 
C

Chris Angelico

I guess it might mean someone decided to config sshd with
PermitRootLogin no... I believe this is common? I don't think it's a
particularly good idea, especially for a large scale deployment.

So I guess there may be some config needed for the machines before
they can be remotely administrated in an automatic fashion.

Depending on what exactly is needed, it might be easier to run a
separate daemon on the computers, one whose sole purpose is to do the
task / get the statistics needed and return them. Then the Python
script need only collect each program's returned response.

Alternatively, if the program needs to be run periodically anyway, it
might be easier to simply cron it on every computer it needs to run
on, and then log the results to some central server (maybe a MySQL
database, or something). Then whenever you want stats, you just query
that server.

Chris Angelico
 
A

Anssi Saari

Chris Angelico said:
Depending on what exactly is needed, it might be easier to run a
separate daemon on the computers, one whose sole purpose is to do the
task / get the statistics needed and return them. Then the Python
script need only collect each program's returned response.

Those would still need to be deployed somehow to the thousands of
machines though.

I realized after posting that something like pexpect might work for
stuffing the keystrokes needed to root login via ssh to all machines
and such... If that's what he needs to do, since it wasn't very clear.
 
B

Babu

Am 07.04.2011 21:14, schrieb Anssi Saari:



But only once...


Maybe that works. But it is much, much worse than using keys...

Thomas

Thank you all for various ideas. Let me give some background and more
information here. Reason that we cannot use root trusted ssh is a
Internal Information Security decision. Given that we have this
restriction, I wanted to explore what other creative options we have
so that we can still accomplish this.

In our enterprise environment, quick production support is very
important. An application problem troubleshooting might require
we check various status on multiple servers quickly. So we need to
execute commands depending on the situation. Let me summarize some of
the ideas presented in this thread.
1. Use pexpect to login and become root(or sudo - yes sudo is
allowed) on the remote machines
2. run a daemon on each server, which will respond to client
requests
3. run your program through cron and collect data and dump into a
database which can be used for query later [ yes - this is on
plate ]
4. Use fabric (fabile.org) for developing program. Does this assume
that ssh root trust is already in place?

Are there any more different approaches? I suppose if we take the
daemon approach then we can make it as a webservice as well?
 
C

Chris Angelico

Are there any more different approaches?  I suppose if we take the
daemon approach then we can make it as a webservice as well?

Yes, your daemon could function via HTTP. But if you go that route,
you would need some way to collect all the different computers'
results.

For example, suppose you build your daemon to respond to HTTP requests
on port 8000, with a document name like "/status". You could then
retrieve _one_ computer's status by pointing your browser to
http://computername/status - but that's only one. You would then need
a wrapper somewhere to collect them, for instance:


<iframe src="http://computer1/status"></iframe>
<iframe src="http://computer2/status"></iframe>
<iframe src="http://computer3/status"></iframe>

etc. If you're always getting status on the same set of computers (or
a few standard sets of computers), this could be a simple .HTML file
that you have on your hard disk; otherwise, you may want to consider
another web server that lets you tick which ones to query, and builds
an iframe list from your selections.

Chris Angelico
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top