Roundup, smtplib, TLS and MS Exchange

[carlistixx]

Hi,

I'm using the roundup issue tracker (http://roundup.sourceforge.net)
which uses smtplib to send mail. It all worked until we moved to a
hosted Exchange MTA. The hosting provider requires the use of TLS. Now
roundup can't send mail.

My version of python is:
Python 2.3.4 (#1, Feb 6 2006, 10:38:46)
[GCC 3.4.5 20051201 (Red Hat 3.4.5-2)] on linux2

Roundup version:
$ roundup-server --version
1.1.2 (python 2.3.4)

I've reproduced the SMTP conversation below. I'm not sure if it's a
problem with Roundup, smtplib, or Exchange.

Any assistance appreciated.

[foobar@moe tracker]$ roundup-server -p 8081
roundup=/home/foobar/roundup/tracker
Roundup server started on :8081
send: 'STARTTLS\r\n'
reply: '503 5.5.2 Send hello first\r\n'
reply: retcode (503); Msg: 5.5.2 Send hello first
send: 'ehlo moe.foobar.local\r\n'
reply: '250-smtpx15.msoutlookonline.net Hello [202.173.131.223]\r\n'
reply: '250-SIZE 31457280\r\n'
reply: '250-PIPELINING\r\n'
reply: '250-ENHANCEDSTATUSCODES\r\n'
reply: '250-STARTTLS\r\n'
reply: '250-AUTH LOGIN\r\n'
reply: '250-8BITMIME\r\n'
reply: '250-BINARYMIME\r\n'
reply: '250 CHUNKING\r\n'
reply: retcode (250); Msg: smtpx15.msoutlookonline.net Hello [202.173.131.223]
SIZE 31457280
PIPELINING
ENHANCEDSTATUSCODES
STARTTLS
AUTH LOGIN
8BITMIME
BINARYMIME
CHUNKING
send: 'AUTH LOGIN ************==\r\n'
reply: '334 ************\r\n'
reply: retcode (334); Msg: ************
send: '************\r\n'
reply: '235 2.7.0 Authentication successful\r\n'
reply: retcode (235); Msg: 2.7.0 Authentication successful
send: 'mail FROM:<[email protected]> size=1090\r\n'
reply: '451 5.7.3 Must issue a STARTTLS command first\r\n'
reply: retcode (451); Msg: 5.7.3 Must issue a STARTTLS command first
send: 'rset\r\n'
send: 'STARTTLS\r\n'
reply: '503 5.5.2 Send hello first\r\n'
reply: retcode (503); Msg: 5.5.2 Send hello first
send: 'ehlo moe.foobar.local\r\n'
reply: '250-smtpx15.msoutlookonline.net Hello [202.173.131.223]\r\n'
reply: '250-SIZE 31457280\r\n'
reply: '250-PIPELINING\r\n'
reply: '250-ENHANCEDSTATUSCODES\r\n'
reply: '250-STARTTLS\r\n'
reply: '250-AUTH LOGIN\r\n'
reply: '250-8BITMIME\r\n'
reply: '250-BINARYMIME\r\n'
reply: '250 CHUNKING\r\n'
reply: retcode (250); Msg: smtpx15.msoutlookonline.net Hello [202.173.131.223]
SIZE 31457280
PIPELINING
ENHANCEDSTATUSCODES
STARTTLS
AUTH LOGIN
8BITMIME
BINARYMIME
CHUNKING
send: 'AUTH LOGIN ******************************==\r\n'
reply: '334 ************\r\n'
reply: retcode (334); Msg: ************
send: '************\r\n'
reply: '235 2.7.0 Authentication successful\r\n'
reply: retcode (235); Msg: 2.7.0 Authentication successful
send: 'mail FROM:<[email protected]> size=1711\r\n'
reply: '451 5.7.3 Must issue a STARTTLS command first\r\n'
reply: retcode (451); Msg: 5.7.3 Must issue a STARTTLS command first
send: 'rset\r\n'
192.168.100.68 - - [31/May/2007 16:14:01] "POST /roundup/issue1638
HTTP/1.1" 400 -
EXCEPTION AT Thu May 31 16:14:01 2007
Traceback (most recent call last):
File "/usr/lib/python2.3/site-packages/roundup/scripts/roundup_server.py",
line 106, in run_cgi
self.inner_run_cgi()
File "/usr/lib/python2.3/site-packages/roundup/scripts/roundup_server.py",
line 266, in inner_run_cgi
tracker.Client(tracker, self, env).main()
File "/usr/lib/python2.3/site-packages/roundup/cgi/client.py", line
196, in main
self.inner_main()
File "/usr/lib/python2.3/site-packages/roundup/cgi/client.py", line
314, in inner_main
self.mailer.exception_message()
File "/usr/lib/python2.3/site-packages/roundup/mailer.py", line 153,
in exception_message
self.standard_message(to, subject, content)
File "/usr/lib/python2.3/site-packages/roundup/mailer.py", line 93,
in standard_message
self.smtp_send(to, message)
File "/usr/lib/python2.3/site-packages/roundup/mailer.py", line 180,
in smtp_send
raise MessageSendError("Error: couldn't send email: %s"%msg)
MessageSendError: Error: couldn't send email: Connection unexpectedly closed

 

[Tijs]

[foobar@moe tracker]$ roundup-server -p 8081
roundup=/home/foobar/roundup/tracker
Roundup server started on :8081
send: 'STARTTLS\r\n'
reply: '503 5.5.2 Send hello first\r\n'
reply: retcode (503); Msg: 5.5.2 Send hello first

I think this must be an issue with roundup, issuing commands in the wrong
order. The correct order should (according to the rfc and the python docs)
be:
* ehlo()
* starttls()
* ehlo()
* login()
....

 

[carlistixx]

Thanks for your response. I tried modifying the Roundup code as suggested:

class SMTPConnection(smtplib.SMTP):
''' Open an SMTP connection to the mailhost specified in the config
'''
def __init__(self, config):

smtplib.SMTP.__init__(self, config.MAILHOST)
self.set_debuglevel(1)
# start the TLS if requested
if config["MAIL_TLS"]:
self.ehlo()
self.starttls(config["MAIL_TLS_KEYFILE"],
config["MAIL_TLS_CERTFILE"])
self.ehlo()

# ok, now do we also need to log in?
mailuser = config["MAIL_USERNAME"]
if mailuser:
self.login(mailuser, config["MAIL_PASSWORD"])

I've inserted a call to ehlo() before and after the call to
starttls(). Adding the first call to ehlo() does help - I don't get so
many complaints from the server. But it still doesn't work- it seems
like the server stops responding after the call to starttls().

I have not specified a keyfile or certfile in the Roundup
configuration, but as I understand it these are optional, so I presume
this is not causing the problem.

[foobar@moe tracker]$ roundup-server -p 8081
roundup=/home/foobar/roundup/tracker
Roundup server started on :8081
send: 'ehlo moe.foobar.local\r\n'
reply: '250-smtpx15.msoutlookonline.net Hello [202.173.131.223]\r\n'
reply: '250-SIZE 31457280\r\n'
reply: '250-PIPELINING\r\n'
reply: '250-ENHANCEDSTATUSCODES\r\n'
reply: '250-STARTTLS\r\n'
reply: '250-AUTH LOGIN\r\n'
reply: '250-8BITMIME\r\n'
reply: '250-BINARYMIME\r\n'
reply: '250 CHUNKING\r\n'
reply: retcode (250); Msg: smtpx15.msoutlookonline.net Hello [202.173.131.223]
SIZE 31457280
PIPELINING
ENHANCEDSTATUSCODES
STARTTLS
AUTH LOGIN
8BITMIME
BINARYMIME
CHUNKING
send: 'STARTTLS\r\n'
reply: '220 2.0.0 SMTP server ready\r\n'
reply: retcode (220); Msg: 2.0.0 SMTP server ready
send: 'ehlo moe.foobar.local\r\n'
send: 'ehlo moe.foobar.local\r\n'
reply: '250-smtpx15.msoutlookonline.net Hello [202.173.131.223]\r\n'
reply: '250-SIZE 31457280\r\n'
reply: '250-PIPELINING\r\n'
reply: '250-ENHANCEDSTATUSCODES\r\n'
reply: '250-STARTTLS\r\n'
reply: '250-AUTH LOGIN\r\n'
reply: '250-8BITMIME\r\n'
reply: '250-BINARYMIME\r\n'
reply: '250 CHUNKING\r\n'
reply: retcode (250); Msg: smtpx15.msoutlookonline.net Hello [202.173.131.223]
SIZE 31457280
PIPELINING
ENHANCEDSTATUSCODES
STARTTLS
AUTH LOGIN
8BITMIME
BINARYMIME
CHUNKING
send: 'STARTTLS\r\n'
reply: '220 2.0.0 SMTP server ready\r\n'
reply: retcode (220); Msg: 2.0.0 SMTP server ready
send: 'ehlo moe.foobar.local\r\n'
192.168.100.68 - - [04/Jun/2007 10:43:24] "POST /roundup/issue1638
HTTP/1.1" 400 -
EXCEPTION AT Mon Jun 4 10:43:24 2007
Traceback (most recent call last):
File "/usr/lib/python2.3/site-packages/roundup/scripts/roundup_server.py",
line 106, in run_cgi
self.inner_run_cgi()
File "/usr/lib/python2.3/site-packages/roundup/scripts/roundup_server.py",
line 266, in inner_run_cgi
tracker.Client(tracker, self, env).main()
File "/usr/lib/python2.3/site-packages/roundup/cgi/client.py", line
196, in main
self.inner_main()
File "/usr/lib/python2.3/site-packages/roundup/cgi/client.py", line
314, in inner_main
self.mailer.exception_message()
File "/usr/lib/python2.3/site-packages/roundup/mailer.py", line 153,
in exception_message
self.standard_message(to, subject, content)
File "/usr/lib/python2.3/site-packages/roundup/mailer.py", line 93,
in standard_message
self.smtp_send(to, message)
File "/usr/lib/python2.3/site-packages/roundup/mailer.py", line 173,
in smtp_send
smtp = SMTPConnection(self.config)
File "/usr/lib/python2.3/site-packages/roundup/mailer.py", line 194,
in __init__
self.ehlo()
File "/usr/lib/python2.3/smtplib.py", line 390, in ehlo
(code,msg)=self.getreply()
File "/usr/lib/python2.3/smtplib.py", line 344, in getreply
line = self.file.readline()
File "/usr/lib/python2.3/smtplib.py", line 159, in readline
chr = self.sslobj.read(1)
sslerror: The read operation timed out

[foobar@moe tracker]$ roundup-server -p 8081
roundup=/home/foobar/roundup/tracker
Roundup server started on :8081
send: 'STARTTLS\r\n'
reply: '503 5.5.2 Send hello first\r\n'
reply: retcode (503); Msg: 5.5.2 Send hello first

I think this must be an issue with roundup, issuing commands in the wrong
order. The correct order should (according to the rfc and the python docs)
be:
* ehlo()
* starttls()
* ehlo()
* login()
...