simple script to read and output Mailbox body to file.

Discussion in 'Python' started by Chuck Amadi, Jun 7, 2004.

  1. Chuck Amadi

    Chuck Amadi Guest

    Sorry to bovver you again (again) here's script.

    I still can't see why the get_payload() doesn't produce
    the plain text message body of an emails in the testwwws users mailbox.
    As you can see I have tried a few things but no joy what am I missing.

    Is the another snippet in relation to get_payload to access the body contents
    print and process to a file.

    Cheers

    Chuck

    ds9:[pythonScriptMail] % cat getSurveyMail.py
    ###############################################################
    ## This script will open and parse email messages body content.
    ## This Python script will reside on Mail Server on ds9:
    ## Emails are all plain/text you could just write the following
    ## Which will leave a list of strings , each one a message body.
    ## The Survey User is testwws and the .procmailrc file folder is
    ## Survey . i.e /home/testwws/Mail/inbox/Survey .
    ###############################################################
    ## file:getSurveyMail.py Created : 06/06/04 Amended date: 07/06/04
    ###############################################################

    #The following line makes it run itself(executable script on UN*X)
    #!/usr/bin/env python

    import sys
    import os
    import email
    import mailbox

    # Open the testwws user mailbox (tmp user chuck)
    # fp denotes factory paraemeter
    # mode can be 'r' when the file will only be read, 'w' for only writing
    #(an existing file with the same name will be erased), and 'a' opens the file
    # for appending; any data written to the file is automatically added to the
    end.
    # 'r+' opens the file for both reading and writing. The mode.
    output =("/tmp/SurveyResults", "w+a")
    #output =('/tmp/SurveyResults','w')

    # open() returns a file object, and is most commonly used with two arguments:
    # "open(filename, mode)".
    # /home/testwwws/Mail/work
    #
    # fp The file or file-like object passed at instantiation time. This can be
    # used to read the message content.
    fp = open("/var/spool/mail/testwwws")

    #fp = open("/home/testwwws/Mail/work")

    # message_from_file returns a message object struct tree from an
    # open file object.

    mbox = mailbox.UnixMailbox(fp, email.message_from_file)
    # list of body messages.
    bodies = []

    msg = email.message_from_file(fp)
    # for loop iterates through the msg in the mbox(mailbox).
    # Subparts of messages can be accessed via the -
    # get_payload() method will return a string object.
    # If it is multipart, use the "walk" method to iterate through each part and
    the
    # get the payload.In our case it's not multipart So ignore.
    # for part in msg.walk():
    # msg = part.get_payload()
    # # do something(print)

    for msg in mbox:
    body = msg.get_payload()
    bodies.append(body)
    # output.close() to close it and free up any system resources taken up by the
    open file.
    # After calling output.close(), attempts to use the file object will
    automatically fail.
    #print bodies
    print fp
    print msg
    print msg['body']
    # print body - NameError: name 'msg' is not defined
    #
    #print >> output,bodies
    #output.close()
    #print the bodies list of the messages.
    print bodies
     
    Chuck Amadi, Jun 7, 2004
    #1
    1. Advertising

  2. Chuck Amadi

    William Park Guest

    Chuck Amadi <> wrote:
    > Sorry to bovver you again (again) here's script.
    >
    > I still can't see why the get_payload() doesn't produce
    > the plain text message body of an emails in the testwwws users mailbox.
    > As you can see I have tried a few things but no joy what am I missing.
    >
    > Is the another snippet in relation to get_payload to access the body
    > contents print and process to a file.

    [snip]

    You still haven't answered central questions.
    - Do you want email bodies in separate files, or all email bodies in
    one file?
    - Do you want to collect these email bodies as they come in, or
    periodically from a 'mbox' file?

    --
    William Park, Open Geometry Consulting, <>
    No, I will not fix your computer! I'll reformat your harddisk, though.
     
    William Park, Jun 7, 2004
    #2
    1. Advertising

  3. Chuck Amadi

    fishboy Guest

    On Mon, 07 Jun 2004 17:20:42 +0100, Chuck Amadi <>
    wrote:

    >fp = open("/var/spool/mail/testwwws")
    >
    >#fp = open("/home/testwwws/Mail/work")
    >
    ># message_from_file returns a message object struct tree from an
    ># open file object.
    >
    >mbox = mailbox.UnixMailbox(fp, email.message_from_file)
    ># list of body messages.
    >bodies = []
    >
    >msg = email.message_from_file(fp)
    ># # do something(print)
    >
    >for msg in mbox:
    > body = msg.get_payload()
    > bodies.append(body)


    Trimmed out a few comments

    Ok, you dont need to pass UnixMailbox email.message_from_file if your
    going to explictly parse later with msg = email.message_from_file(fp)

    Your parsing the email message twice.

    If there are still problems, try breaking it down a bit

    mbox = mailbox.UnixMailbox(fp)
    for mail in mbox:
    print mail.read()
    break #just look at one message

    'mail' is a file obj pointing at a single email, so print mail.read()
    should display one email, with headers. If that doesn't work we have
    problems before we get anywhere

    If that gives us something sensible for an output, try parsing the
    email.

    mbox = mailbox.UnixMailbox(fp)
    for mail in mbox:
    msg = email.message_from_file(mail)
    print `msg` #backticks
    print msg['Subject']
    print msg.get_content_type()
    break #just look at one message

    Should give us something like:

    <email.Message.Message instance at 0xa0e442c>
    Re: Best IDE?
    text/plain

    And if that works, then

    msg.get_payload() should return the body.

    ><{{{*>
     
    fishboy, Jun 7, 2004
    #3
  4. Chuck Amadi

    chuck amadi Guest

    William Park wrote:

    >Chuck Amadi <> wrote:
    >
    >
    >>Sorry to bovver you again (again) here's script.
    >>
    >>I still can't see why the get_payload() doesn't produce
    >>the plain text message body of an emails in the testwwws users mailbox.
    >>As you can see I have tried a few things but no joy what am I missing.
    >>
    >>Is the another snippet in relation to get_payload to access the body
    >>contents print and process to a file.
    >>
    >>

    >[snip]
    >
    >You still haven't answered central questions.
    > - Do you want email bodies in separate files, or all email bodies in
    > one file?
    > - Do you want to collect these email bodies as they come in, or
    > periodically from a 'mbox' file?
    >
    >
    >

    Hi There ,

    I would like all email bodies in one file.Collect these email bodies as they come in, or periodically from a 'mbox' file? using cron.

    I would lie cron to run the script and empty the mail user box in question to a file for extraction to a database.

    Cheers

    Chuck
     
    chuck amadi, Jun 8, 2004
    #4
  5. Chuck Amadi

    Chuck Amadi Guest

    I have tried breaking it down a bit as per your post.
    when I run it I still dont see any results even to my output file .
    Im logged in as myself ansd trying to access the mailbox on the mailserver
    which is mounted via nfs.The mail server has only version Python 1.5 So doesnt
    know about the email module.

    Cheers

    output =('/tmp/SurveyResults','w')

    fp = open("/var/spool/mail/chuck")


    mbox = mailbox.UnixMailbox(fp)
    for mail in mbox:
    print mail.read()
    break #just look at one message
     
    Chuck Amadi, Jun 8, 2004
    #5
  6. Chuck Amadi

    fishboy Guest

    On Tue, 08 Jun 2004 09:49:41 +0100, Chuck Amadi <>
    wrote:

    >I have tried breaking it down a bit as per your post.
    >when I run it I still dont see any results even to my output file .
    >Im logged in as myself ansd trying to access the mailbox on the mailserver
    >which is mounted via nfs.The mail server has only version Python 1.5 So doesnt
    >know about the email module.
    >
    >Cheers
    >
    >output =('/tmp/SurveyResults','w')
    >
    >fp = open("/var/spool/mail/chuck")
    >
    >
    >mbox = mailbox.UnixMailbox(fp)
    >for mail in mbox:
    > print mail.read()
    > break #just look at one message
    >
    >


    What does a
    $less /var/spool/mail/chuck
    give you?

    UnixMailbox is expecting a single file with all the emails
    concatenated in it. UnixMailbox just splits them out one at a time.

    Hrmmm, looking in the source, it recommends that you use
    PortableUnixMailbox instead. If you can see the emails with the
    'less' command, maybe that will work.

    If you don't see any emails with 'less', I'm left wondering where your
    emails are at?

    ><{{{*>
     
    fishboy, Jun 8, 2004
    #6
  7. Chuck Amadi

    William Park Guest

    chuck amadi <> wrote:
    > William Park wrote:
    > >You still haven't answered central questions.
    > > - Do you want email bodies in separate files, or all email bodies in
    > > one file?
    > > - Do you want to collect these email bodies as they come in, or
    > > periodically from a 'mbox' file?
    > >

    > Hi There ,
    >
    > I would like all email bodies in one file.Collect these email bodies
    > as they come in, or periodically from a 'mbox' file? using cron.
    >
    > I would lie cron to run the script and empty the mail user box in
    > question to a file for extraction to a database.


    Since your running Linux, you can simply do manual or periodic parsing
    of mbox file and put the email bodies into 'file',
    formail -I '' -s < mbox >> file
    Or, use Procmail recipe as email comes in,
    :0 b:
    * some_condition
    | cat >> file

    Ref:
    man formail procmail procmailrc procmailex

    Homework:
    Translate it to Python.

    --
    William Park, Open Geometry Consulting, <>
    No, I will not fix your computer! I'll reformat your harddisk, though.
     
    William Park, Jun 8, 2004
    #7
  8. Chuck Amadi

    Chuck Amadi Guest

    Hi again I using this to test as instructed it runs on my local machine with
    my /var/spool/mail/chuck dir
    thats mounted on the mail server using nfs .
    Thus I still get no output only when I print mbox.

    NB Im digesting formail doc's and man pages.

    Cheers

    Chuck

    #!/usr/bin/env python


    import email
    import mailbox


    fp =open("/var/spool/mail/chuck")
    mbox = mailbox.UnixMailbox(fp)


    for mail in mbox:
    print mail.read()
    break # Just reads one message
     
    Chuck Amadi, Jun 8, 2004
    #8
  9. Chuck Amadi

    Chuck Amadi Guest

    Nice One I was thinking I was going mad at home this works .
    But $ less /var/spool/mail/chuck produced no output.

    I use scan+ to check and go thru users email folder . i.e

    cd /home/User/Mail/Inbox <Folder in Question> say spam Folder

    > scan +Lottery > show 1 example below:


    chuck@sevenofnine:~/pythonScript> cd /home/chuck/Mail
    chuck@sevenofnine:~/Mail> ls
    :0 Chuck drafts Inbox mhnjVMtAk pete post2ic2cT post57nXqr
    postG9QD2D postW78Qle Root spam
    Beth chucksMail from mbox Operator Pete post2iybnf post8ReyCU
    posthArGZR postz5FN1w rootMail trash
    chuck context inbox mhn9VVBQf outbox Pick6 post4XPUvv postc6DZgu
    postnnglMC Rhys sent-mail
    chuck@sevenofnine:~/Mail> scan +spam
    1 06/08 Woolwich Internet Attention aII Woolwich clients.<<This is a
    multi-part message in MIME format.
    chuck@sevenofnine:~/Mail> show 1
    (Message spam:1)
    part 1 text/plain

    Does this explain my delemour.We use procmail , fetchmail and sendmail

    Cheers
     
    Chuck Amadi, Jun 8, 2004
    #9
  10. Chuck Amadi

    Chuck Amadi Guest

    Hi again yes the /var/spool/mail/chuck is empty as we process the mail to the
    user home dir using MH .
    Thus created a mailbox using packf with three emails for testing .

    Cheers for pointing this out.
     
    Chuck Amadi, Jun 8, 2004
    #10
  11. Chuck Amadi

    Chuck Amadi Guest

    Hi using the breakdown script I managed this minis mail.read() just print mail.

    mbox = mailbox.UnixMailbox(fp)
    for mail in mbox:
    print mail
    break #just look at one message

    Here's the output of the script. Cheers chuck

    chuck@sevenofnine:~/pythonScript> python testmail.py
    Return-Path: <>
    Received: from enterprise.smtl.co.uk (enterprise.smtl.co.uk [193.131.77.1***])
    by ds9.smtl.co.uk (8.12.10/8.11.0) with ESMTP id i4PDJXgN006468
    for <>; Tue, 25 May 2004 14:19:35 +0100
    Message-Id: <>
    Subject: WWW Survey
    To:
    From:
    Date: Tue, 25 May 2004 14:19:14 +0100
    X-Spam-Checker-Version: SpamAssassin 2.63 (2004-01-11) on ds9.smtl.co.uk
    X-Spam-Status: No, hits=0.3 required=5.0 tests=NO_REAL_NAME autolearn=no
    version=2.63
    X-Spam-Level:

    chuck@sevenofnine:~/pythonScript>
     
    Chuck Amadi, Jun 8, 2004
    #11
  12. Chuck Amadi

    Chuck Amadi Guest

    This now works ! as you stated below.Now all the body messages is this down
    using bodies [] list.

    Cheers Chuck

    mbox = mailbox.UnixMailbox(fp)
    for mail in mbox:
    msg = email.message_from_file(mail)
    print `mail` #backticks
    print mail['Subject']
    print mailget_content_type()
    break #just look at one message

    Gives me this:

    chuck@sevenofnine:~/pythonScript> python getSurveyMail.py
    mail
    WWW Survey
    text/plain
    <wwwsurvey>
    Country you are currently working in:<country>UNITED STATES</country>
    Postal code or ZIP Code of your facility if you know it:<zipcode>
    12345</zipcode>
    Your Profession:<profession>Nurse</profession>
    How long you have been qualifed:<qualified>&lt; 1 Year</qualified>
    Is Wound Management Your primary role:<iswoundmgt>A major part of your daily
    activities</iswoundmgt>
    Specific area of interest in the list below:<specificareas>Veterinary
    medicine</specificareas>
    Do you read a paper &ndash; based wm journal:<journal>Yes</journal>
    If Yes your comments:<jcomments>bugs news</jcomments>
    Favorite topics in the World Wide Wounds:<favorite>['Dressings information',
    'News']</favorite>
    Further Information: required:<furtherinfo>all covered really</furtherinfo>
    Your Email Address:<email></email>
    </wwwsurvey>


    that works, then I done this

    msg.get_payload() should return the body. How about all the body messages in
    file.

    Thanks
     
    Chuck Amadi, Jun 8, 2004
    #12
  13. Chuck Amadi

    Chuck Amadi Guest

    Hi again after breaking down the script into easier chunks .
    I now get this .

    Here's the output from the script but WHAT about all messages bodies

    Cheers

    chuck@sevenofnine:~/pythonScript> python getSurveyMail.py
    mail
    WWW Survey
    text/plain
    <wwwsurvey>
    Country you are currently working in:<country>UNITED STATES</country>
    Postal code or ZIP Code of your facility if you know it:<zipcode>
    12345</zipcode>
    Your Profession:<profession>Nurse</profession>
    How long you have been qualifed:<qualified>&lt; 1 Year</qualified>
    Is Wound Management Your primary role:<iswoundmgt>A major part of your daily
    activities</iswoundmgt>
    Specific area of interest in the list below:<specificareas>Veterinary
    medicine</specificareas>
    Do you read a paper &ndash; based wm journal:<journal>Yes</journal>
    If Yes your comments:<jcomments>bugs news</jcomments>
    Favorite topics in the World Wide Wounds:<favorite>['Dressings information',
    'News']</favorite>
    Further Information: required:<furtherinfo>all covered really</furtherinfo>
    Your Email Address:<email></email>
    </wwwsurvey>
     
    Chuck Amadi, Jun 8, 2004
    #13
  14. Chuck Amadi

    Chuck Amadi Guest

    Hi After createing a mailbox using packf
    packf +/home/chuck/testwwws all -file testbox and thus cp 3 5 6 (emails) .

    I get output with this except only one email header and body message , the
    commented out stuff , does that iterate or can it produce all current 3 email
    body messages. Cheers Im nearly there.

    for mail in mbox:
    # body = mail.get_payload()
    # bodies.append(body)
    # msg = mail.message_from_file(mail)
    print 'mail'
    print mail['Subject']
    print mail.get_content_type()#text/plain
    print mail.get_payload()
    break # just look at one message
     
    Chuck Amadi, Jun 9, 2004
    #14
  15. Chuck Amadi

    Chuck Amadi Guest

    Hi all forget my last post about all email body messages I added a comment
    before break . Sorry for reading my own comments lines.

    I get output desired .Now to pass to an external File.

    for mail in mbox:
    # body = mail.get_payload()
    # bodies.append(body)
    # msg = mail.message_from_file(mail)
    print 'mail'
    print mail['Subject']
    print mail.get_content_type()#text/plain
    print mail.get_payload()
    # break # just look at one message
     
    Chuck Amadi, Jun 9, 2004
    #15
  16. Chuck Amadi

    Chuck Amadi Guest

    Hi again

    I get output desired .Now to parse the fileobject and all body messages to an
    external File named SurveyResults.txt
    Is this the correct way to go about it .

    for mail in mbox:
    # body = mail.get_payload()
    # bodies.append(body)
    # msg = mail.message_from_file(mail)
    print 'mail'
    print mail['Subject']
    print mail.get_content_type()#text/plain
    print mail.get_payload()
    # break # just look at one message

    # The File SurveyResults.txt must all ready exist.
    output = open("SurveyResults.txt", 'w') # mode 'w' means open the file for
    writ-ing (any data already in the file will be erased)
    mailout = mail.get_payload()
    output. writelines(mailout)

    output. close() # this is at end of the script

    Here's my error but I thought mail is my file

    Traceback (most recent call last):
    File "getSurveyMailRev2.py", line 37, in ?
    mailout = mail.get_payload()
    NameError: name 'mail' is not defined
     
    Chuck Amadi, Jun 9, 2004
    #16
  17. Chuck Amadi

    Chuck Amadi Guest

    Hi please could someone point to what I have done wrong as the script runs
    with no errors but the file I am trying to parse the print statement output
    too instead of just the screen/command line to another file.

    I have digested the Input and Output and text formatting docs and I read
    something about using stdout to a file . Can someone point me in the right
    direction.

    Cheers my code below.

    ###############################################################
    ## This script will open and parse email messages body content.
    ## This Python script will reside on Mail Server on ds9:
    ## Emails are all plain/text you could just write the following
    ## Which will leave a list of strings , each one a message body.
    ## The Survey User is testwws and the .procmailrc file folder is
    ## Survey . i.e /home/testwws/Mail/inbox/Survey .
    ###############################################################
    ## file:getSurveyMail.py Created : 06/06/04 Amended date: 09/06/04
    ## Revision 1. make copy to continue called getSurveyMailRev2.py
    ###############################################################

    #The following line makes it run itself(executable script on UN*X)
    #!/usr/bin/env python

    import sys
    import os
    import email
    import mailbox
    import StringIO

    # Open the testwws user mailbox (tmp user chuck)
    # fp denotes factory paraemeter
    # mode can be 'r' when the file will only be read, 'w' for only writing
    #(an existing file with the same name will be erased), and 'a' opens the file
    # for appending; any data written to the file is automatically added to the end.
    # 'r+' opens the file for both reading and writing. The mode.

    # The File SurveyResults.txt must all ready exist.

    #mailout = open("/home/chuck/pythonScript/SurveyResults.txt") # mode 'w' means open the file for writ-ing
    # (any data already in the file will be erased)
    #output.writelines(mailout)
    mailout = file("/home/chuck/pythonScript/SurveyResults.txt")

    # open() returns a file object, and is most commonly used with two arguments:
    # "open(filename, mode)".
    fp = open("/home/chuck/pythonScript/testbox")

    # message_from_file returns a message object struct tree from an
    # open file object.

    mbox = mailbox.UnixMailbox(fp, email.message_from_file)
    # list of body messages.
    bodies = []

    # mail is the file object
    for mail in mbox:
    print 'mail'
    print mail['Subject']
    print mail.get_content_type()#text/plain
    print mail.get_payload()


    # First open the testbox file to read(r) and write(w)to the SurveyResults.txt
    #fp = open("testbox","r")
    mailout = open("/home/chuck/pythonScript/SurveyResults.txt","w")

    # Read the testbox file into a list then copy to
    # new file.
    for mail in fp.readlines():
    mailout.write(mail)

    print "testbox file copied...to SurveyResults.txt"

    # Now close the files
    fp.close()
    mailout.close()

    #mailout.close to close it and free up any system resources taken up by the open file.
    # After calling output.close(), attempts to use the file object will automatically fail.
     
    Chuck Amadi, Jun 9, 2004
    #17
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Chuck Amadi
    Replies:
    4
    Views:
    651
    William Park
    Jun 6, 2004
  2. chuck amadi

    simple script to read and parse mailbox

    chuck amadi, Jun 5, 2004, in forum: Python
    Replies:
    6
    Views:
    510
    Chuck Amadi
    Jun 7, 2004
  3. Chuck Amadi

    simple script to read and parse mailbox

    Chuck Amadi, Jun 7, 2004, in forum: Python
    Replies:
    0
    Views:
    308
    Chuck Amadi
    Jun 7, 2004
  4. Chuck Amadi

    simple script to read and parse mailbox

    Chuck Amadi, Jun 7, 2004, in forum: Python
    Replies:
    0
    Views:
    621
    Chuck Amadi
    Jun 7, 2004
  5. London Boy
    Replies:
    2
    Views:
    359
    Keith Bowes
    Jan 12, 2004
Loading...

Share This Page