python backup script

[MMZ]

I am trying to backup database on CentOS linux server,I'm getting error when running the following script. anyone can help?


#!/usr/bin/env python
import ConfigParser
import os
import time

config = ConfigParser.ConfigParser()
config.read("~/my.cnf")
username = config.get('client', 'mmz')
password = config.get('client', 'pass1')
hostname = config.get('client', 'localhost')
filestamp = time.strftime('%Y-%m-%d')

# Get a list of databases with :
Database_list_command="mysql -u %s -p%s -h %s --silent -N -e 'show databases'" % (username, password, hostname)
for database in os.popen(database_list_command).readlines():
database = database.strip()
if database == 'information_schema':
continue
if database == 'db_dev':
continue
filename = "/backups/mysql/%s-%s.sql" % (database, filestamp)
os.popen("mysqldump -u %s -p%s -h %s -e --opt -c %s | gzip -c > %s.gz" % (username, password, hostname, database, filename))



.............Error..........

Traceback (most recent call last):
File "./backup.py", line 8, in ?
username = config.get('client', 'mmz')
File "/usr/lib/python2.4/ConfigParser.py", line 511, in get
raise NoSectionError(section)

 

[Jerry Hill]

I am trying to backup database on CentOS linux server,I'm getting error
when running the following script. anyone can help?

Traceback (most recent call last):
File "./backup.py", line 8, in ?
username = config.get('client', 'mmz')
File "/usr/lib/python2.4/ConfigParser.py", line 511, in get
raise NoSectionError(section)

​I've never used ConfigParser, but that ​

​error message looks pretty simple to interpret. You've set up a
ConfigParser object, told it to read in ~/my.cnf, the asked for the value
of section 'client', option 'mmz'. The error indicates that your config
files doesn't have a section named 'client'.

What is the content of your ~/my.cnf file?

 

[MMZ]

I am trying to backup database on CentOS linux server,I'm getting error when running the following script. anyone can help?




Traceback (most recent call last):

  File "./backup.py", line 8, in ?

    username = config.get('client', 'mmz')

  File "/usr/lib/python2.4/ConfigParser.py", line 511, in get

    raise NoSectionError(section)



I've never used ConfigParser, but that  

error message looks pretty simple to interpret.  You've set up a ConfigParser object, told it to read in ~/my.cnf, the asked for the value of section 'client', option 'mmz'.  The error indicates that your config files doesn't have a section named 'client'.



What is the content of your ~/my.cnf file?

Thank you for helping Jerry. Actually I found this script for debian but I want to use it for CentOS server so I replaced /etc/mysql/debian.cnf with ~/my.cnf
the file content is:

Example MySQL config file for medium systems.
#
# This is for a system with little memory (32M - 64M) where MySQL plays
# an important part, or systems up to 128M where MySQL is used together with
# other programs (such as a web server)
#
# MySQL programs look for option files in a set of
# locations which depend on the deployment platform.
# You can copy this option file to one of those
# locations. For information about these locations, see:
# http://dev.mysql.com/doc/mysql/en/option-files.html
#
# In this file, you can use all long options that a program supports.
# If you want to know which options a program supports, run the program
# with the "--help" option.

# The following options will be passed to all MySQL clients
[client]
#password = your_password
port = 3306
socket = /tmp/mysql.sock

# Here follows entries for some specific programs

# The MySQL server
[mysqld]
port = 3306
socket = /tmp/mysql.sock
skip-locking
key_buffer_size = 16M
max_allowed_packet = 1M
table_open_cache = 64
sort_buffer_size = 512K
net_buffer_length = 8K
read_buffer_size = 256K
read_rnd_buffer_size = 512K
myisam_sort_buffer_size = 8M

# Don't listen on a TCP/IP port at all. This can be a security enhancement,
# if all processes that need to connect to mysqld run on the same host.
# All interaction with mysqld must be made via Unix sockets or named pipes.
# Note that using this option without enabling named pipes on Windows
# (via the "enable-named-pipe" option) will render mysqld useless!
#
#skip-networking

# Replication Master Server (default)
# binary logging is required for replication
log-bin=mysql-bin

# binary logging format - mixed recommended
binlog_format=mixed

# required unique id between 1 and 2^32 - 1
# defaults to 1 if master-host is not set
# but will not function as a master if omitted
server-id = 1

 

[MMZ]

Thanks Matt.
my.cnf is a readonly file and cannot be changed or modified but do you knowof a file that stores similar information on CentOS?I think I'm not reading from a right file maybe.

I've never used ConfigParser either, but shouldn't the "[client]" sectionhave the options "mmz", "pass1", or "localhost" somewhere?  Do you need to add them to that file?




Matt Jones

I am trying to backup database on CentOS linux server,I'm getting errorwhen running the following script. anyone can help?





Traceback (most recent call last):

  File "./backup.py", line 8, in ?

    username = config.get('client', 'mmz')

  File "/usr/lib/python2.4/ConfigParser.py", line 511, in get

    raise NoSectionError(section)




I've never used ConfigParser, but that  

error message looks pretty simple to interpret.  You've set up a ConfigParser object, told it to read in ~/my.cnf, the asked for the value of section 'client', option 'mmz'.  The error indicates that your config filesdoesn't have a section named 'client'.





What is the content of your ~/my.cnf file?




Jerry

Thank you for helping Jerry. Actually I found this script for debian but I want to use it for CentOS server so I replaced /etc/mysql/debian.cnf with~/my.cnf

the file content is:



 Example MySQL config file for medium systems.

#

# This is for a system with little memory (32M - 64M) where MySQL plays

# an important part, or systems up to 128M where MySQL is used together with

# other programs (such as a web server)

#

# MySQL programs look for option files in a set of

# locations which depend on the deployment platform.

# You can copy this option file to one of those

# locations. For information about these locations, see:

# http://dev.mysql.com/doc/mysql/en/option-files.html

#

# In this file, you can use all long options that a program supports.

# If you want to know which options a program supports, run the program

# with the "--help" option.



# The following options will be passed to all MySQL clients

[client]

#password       = your_password

port            = 3306

socket          = /tmp/mysql.sock



# Here follows entries for some specific programs



# The MySQL server

[mysqld]

port            = 3306

socket          = /tmp/mysql.sock

skip-locking

key_buffer_size = 16M

max_allowed_packet = 1M

table_open_cache = 64

sort_buffer_size = 512K

net_buffer_length = 8K

read_buffer_size = 256K

read_rnd_buffer_size = 512K

myisam_sort_buffer_size = 8M



# Don't listen on a TCP/IP port at all. This can be a security enhancement,

# if all processes that need to connect to mysqld run on the same host.

# All interaction with mysqld must be made via Unix sockets or named pipes.

# Note that using this option without enabling named pipes on Windows

# (via the "enable-named-pipe" option) will render mysqld useless!

#

#skip-networking



# Replication Master Server (default)

# binary logging is required for replication

log-bin=mysql-bin



# binary logging format - mixed recommended

binlog_format=mixed



# required unique id between 1 and 2^32 - 1

# defaults to 1 if master-host is not set

# but will not function as a master if omitted

server-id       = 1

 

[Enrico 'Henryx' Bianchi]

config.read("~/my.cnf")
username = config.get('client', 'mmz')
password = config.get('client', 'pass1')
hostname = config.get('client', 'localhost')

### A simple config file ###
[client]
user = mmz
password = pass1
host = localhost
### EOF ###

#!/usr/bin/env python
import ConfigParser

config = ConfigParser.ConfigParser()
config.read("~/configfile.cfg")
username = config.get('client', 'user')
password = config.get('client', 'password')
hostname = config.get('client', 'host')
[...]

# Get a list of databases with :
Database_list_command="mysql -u %s -p%s -h %s --silent -N -e 'show

databases'" % (username, password, hostname)

for database in os.popen(database_list_command).readlines():
database = database.strip()
if database == 'information_schema':
continue
if database == 'db_dev':
continue
filename = "/backups/mysql/%s-%s.sql" % (database, filestamp)
os.popen("mysqldump -u %s -p%s -h %s -e --opt -c %s | gzip -c > %s.gz"

% (username, password, hostname, database, filename))

command = subprocess.Popen(['mysql', '-u ' + username,
'-p' + password, '-h ' + hostname,
'--silent', '-N', "-e 'show databases'"],
shell=False,
stdout=subprocess.PIPE)
status = command.wait()
for line in command.stdout.readlines():
if line.strip() not in ['information_schema', 'db_dev']:
filename = "/backups/mysql/%s-%s.sql" % (database, filestamp)
cmd1 = subprocess.Popen(['mysqldump', '-u ' + username,
'-p' + password, '-h ' + hostname,
'-e', '--opt', '-c ' + database],
shell=False,
stdout=subprocess.PIPE)
cmd2 = subprocess.Popen(['gzip' '-c'],
shell=False,
stdout=filename)


Enrico

 

[Enrico 'Henryx' Bianchi]

cmd2 = subprocess.Popen(['gzip' '-c'],
shell=False,
stdout=filename)

Doh, my fault:

cmd2 = subprocess.Popen(['gzip' '-c'],
shell=False,
stdout=filename
stdin=cmd1.stdout)

Enrico

 

[Chris Angelico]

username = config.get('client', 'mmz')
password = config.get('client', 'pass1')
hostname = config.get('client', 'localhost')

Are 'mmz', 'pass1', and 'localhost' the actual values you want for
username, password, and hostname? If so, don't pass them through
config.get() at all - just use them directly. In fact, I'd be inclined
to just stuff them straight into the Database_list_command literal;
that way, it's clear how they're used, and the fact that you aren't
escaping them in any way isn't going to be a problem (tip: an
apostrophe in your password would currently break your script).

It's also worth noting that the ~/ notation is a shell feature. You
may or may not be able to use it in config.read().

ChrisA

 

[mina]

Enrico 'Henryx' Bianchi wrote:

cmd2 = subprocess.Popen(['gzip' '-c'],

shell=False,

stdout=filename)

Doh, my fault:



cmd2 = subprocess.Popen(['gzip' '-c'],

shell=False,

stdout=filename

stdin=cmd1.stdout)



Enrico

Thank you Enrico. I've just tried your script and got this error:
stdin=cmd1.stdout)
^
SyntaxError: invalid syntax

any idea?

 

[MMZ]

stdin=cmd1.stdout)
^
SyntaxError: invalid syntax

Enrico 'Henryx' Bianchi wrote:

cmd2 = subprocess.Popen(['gzip' '-c'],

shell=False,

stdout=filename)

Doh, my fault:



cmd2 = subprocess.Popen(['gzip' '-c'],

shell=False,

stdout=filename

stdin=cmd1.stdout)



Enrico

 

[John Gordon]

cmd2 = subprocess.Popen(['gzip' '-c'],

shell=False,

stdout=filename

stdin=cmd1.stdout)

Thank you Enrico. I've just tried your script and got this error:
stdin=cmd1.stdout)
^
SyntaxError: invalid syntax

Looks like you need a comma after 'stdout=filename'.

 

[MRAB]

cmd2 = subprocess.Popen(['gzip' '-c'],
shell=False,
stdout=filename)

Doh, my fault:

cmd2 = subprocess.Popen(['gzip' '-c'],
shell=False,
stdout=filename
stdin=cmd1.stdout)

Enrico

Thank you Enrico. I've just tried your script and got this error:
stdin=cmd1.stdout)
^
SyntaxError: invalid syntax

any idea?

Missing comma on the previous line.

 

[MMZ]

Are 'mmz', 'pass1', and 'localhost' the actual values you want for

username, password, and hostname? If so, don't pass them through

config.get() at all - just use them directly. In fact, I'd be inclined

to just stuff them straight into the Database_list_command literal;

that way, it's clear how they're used, and the fact that you aren't

escaping them in any way isn't going to be a problem (tip: an

apostrophe in your password would currently break your script).



It's also worth noting that the ~/ notation is a shell feature. You

may or may not be able to use it in config.read().



ChrisA

Thanks Chris. you are right.
So I used them directly and removed configParser. The new error is:

Traceback (most recent call last):
File "./bbk.py", line 11, in ?
for database in os.popen(database_list_command).readlines():
NameError: name 'database_list_command' is not defined

any idea?

 

[Chris Angelico]

Thanks Chris. you are right.
So I used them directly and removed configParser. The new error is:

Traceback (most recent call last):
File "./bbk.py", line 11, in ?
for database in os.popen(database_list_command).readlines():
NameError: name 'database_list_command' is not defined

Python names are case-sensitive. If you create it with a capital D,
you can't reference it with a lower-case d. For consistency with the
rest of your code, I'd advise lowercasing it.

ChrisA

 

[MRAB]

Thanks Chris. you are right.
So I used them directly and removed configParser. The new error is:

Traceback (most recent call last):
File "./bbk.py", line 11, in ?
for database in os.popen(database_list_command).readlines():
NameError: name 'database_list_command' is not defined

any idea?

Check the spelling (remember that the name is case-sensitive).

 

[Peter Otten]

It's also worth noting that the ~/ notation is a shell feature. You
may or may not be able to use it in config.read().

The latter. Combined with

"""
read(self, filenames) method of ConfigParser.ConfigParser instance
Read and parse a filename or a list of filenames.

Files that cannot be opened are silently ignored; this is
....
"""

you'll get no configuration at all. You have to apply

os.path.expanduser()

to get a valid path.

$ cd
$ touch tmp.config
$ cd /
$ python
Python 2.7.2+ (default, Jul 20 2012, 22:15:08)
[GCC 4.6.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.

from ConfigParser import ConfigParser
p = ConfigParser()
p.read("~/tmp.config") []
import os
p.read(os.path.expanduser("~/tmp.config"))

['/home/peter/tmp.config']

 

[Enrico 'Henryx' Bianchi]

John Gordon wrote:

Looks like you need a comma after 'stdout=filename'.

Sigh, yesterday was a terrible day (yes, it lacks a comma)...
Anyway, when it is possible, is recommended to use the drivers for
communicate with databases, because subprocess (or os.*open*) is more
expensive compared to (python needs to spawn an external process to execute
the command)

Enrico