Inconsistent SMTP/Gmail connection drop

[Astley Le Jasper]

I have a laptop that wakes up and then runs a script that collects
info and sends an email with a spreadsheet (zipped and about 350KB)
report to a number of people. However, every time I get the following
error:


File "/usr/lib/python2.7/smtplib.py", line 343, in getreply
raise SMTPServerDisconnected("Connection unexpectedly closed")


- When I check the laptop and manually run the function that sends the
email and report, it works fine. I have never been able to replicate
the error when doing it manually.
- It can't be a problem with Gmail at that time in the morning or my
connection, because the script also sends me a copy of the log file
straight after, and that works.
- The same code has been working for years, it just that I have
recently rebuilt the machine so now it is using Python 2.7. I have
also change the references to use absolute file references, rather
than relative. I also have changed the code to run as root.
- I've checked the smtp debug report and everything seems ok. I just
never get the confirmation code (250) to say that it's completed.
- Even stranger ... it ran ok once. But I didn't change a thing and
the next day it wouldn't work.

The only thing I can think of is that when the file is transferred to
the reports directory, it somehow isn't been released quickly enough
and perhaps this is tripping up SMTP. But ... this is a bit of a guess
given I've tried to exhaust all the other options.

Any ideas?

 

[Astley Le Jasper]

Not sure how I'd test or determine that. I've set smtp debug to true
and the output looks all good. I suppose I could try sending an email
before the report is sent, as well as the email that is sent after it.

 

[Astley Le Jasper]

Thanks for those suggestions.

I tried something last night before I got your ideas.

1. I added a line to send a copy of the report just to me, 2 lines
before the line where it emails the report to all the recipients.
2. I also added a timer.sleep(5) pause just before the line that
emails the reports to all the recipients.

So the order was:

a) Send the full report just to me .... FAILED.
b) Pause for 5 seconds.
c) Send the full report to all 4 recipients ... WORKED.
d) Send the log file just to me ... WORKED.

So ... what looks like may be happening is that something might still
be locking the file. In which case i'll try again without the test
email to me. And/or perhaps send a test email without the attachment
before the main report to see if that gets through, to see if the
connection with gmail is actually really connected.

See also below...

1) You might see if there's something about the size of the message - is it
bigger after collecting data all night?  Is google disconnecting after a
maximum amount of data is transferred?

I don't think so. The test message I mentioned above got through and
the attachment was identical (320kb or abouts)

2) You might try sending a tiny test message at the beginning, just to
yourself, and seeing if that has the problem

As above.

3) You might try running your script under a system call tracer:
http://stromberg.dnsalias.org/~dstromberg/debugging-with-syscall-tracers.html
- and inspecting the list of syscalls and their return codes near when the
process terminates

Ouch. Looks complicated!
;-)

4) I get disconnects from gmail once in a while too, especially when
downloading a large list of new message headers; I've been just coding
around it by making my code idempotent and otherwise restartable.  This
seems to happen with both 2.x and 3.x, so I assume it's not really related
to the Python version, but I suppose I should try it on PyPy or Jython
someday.  Note that the error message in 3.x was better than in 2.x.

I was thinking that perhaps I could put a loop in that tests if the
email goes through, and if not, pauses for a few seconds and then
tries again.

 

[Astley Le Jasper]

Just to add a little bit to the mix, I have started having these
problems since I rebuilt my machine (Xubuntu 11) and I changed the
disc formats to ext4 without even thinking about it ... I was
wondering if maybe that had something to do with it?