IOError - cannot create file (linux daemon-invoked script)

Discussion in 'Python' started by cassiope, Jan 2, 2010.

  1. cassiope

    cassiope Guest

    I have a daemon on a Linux system that supports a number of Windows
    clients. Among the functions is to send e-mails, which is
    sufficiently complicated that I fork() a separate process which gets
    setuid to a lesser user, and calls a python script which does the
    actual formatting and emailing (the daemon is written in C). I want
    to save a copy of the email in a particular directory which is
    accessible to the Windows clients via samba.

    The strange thing is that even with the right user-id, I cannot seem
    to write to the directory, getting an IOError exception. Changing the
    directory to world-writable fixes this. I can confirm the uid and gid
    for the script by having the script print these values just before
    trying to create/write the file. Becoming the same lesser user, I
    have no problem writing a file to the same directory.

    Is there anything that I can do to diagnose why this script is
    failing? For various reasons I don't want to make the directory world-
    writable.

    This is on a Debian "squeeze" system, with python 2.5.

    Thanks for any insights!
    cassiope, Jan 2, 2010
    #1
    1. Advertising

  2. cassiope

    Steve Holden Guest

    cassiope wrote:
    > I have a daemon on a Linux system that supports a number of Windows
    > clients. Among the functions is to send e-mails, which is
    > sufficiently complicated that I fork() a separate process which gets
    > setuid to a lesser user, and calls a python script which does the
    > actual formatting and emailing (the daemon is written in C). I want
    > to save a copy of the email in a particular directory which is
    > accessible to the Windows clients via samba.
    >
    > The strange thing is that even with the right user-id, I cannot seem
    > to write to the directory, getting an IOError exception. Changing the
    > directory to world-writable fixes this. I can confirm the uid and gid
    > for the script by having the script print these values just before
    > trying to create/write the file. Becoming the same lesser user, I
    > have no problem writing a file to the same directory.
    >

    Have you looked at the IOError's errno attribute to find out exactly why
    the Python subprocess is unable to write to the directory?

    > Is there anything that I can do to diagnose why this script is
    > failing? For various reasons I don't want to make the directory world-
    > writable.
    >

    I'd concur on that decision.

    > This is on a Debian "squeeze" system, with python 2.5.
    >
    > Thanks for any insights!


    Take a closer look at the exception, that might stimulate a thought or two.

    regards
    Steve
    --
    Steve Holden +1 571 484 6266 +1 800 494 3119
    PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/
    Holden Web LLC http://www.holdenweb.com/
    UPCOMING EVENTS: http://holdenweb.eventbrite.com/
    Steve Holden, Jan 2, 2010
    #2
    1. Advertising

  3. On 02Jan2010 15:21, cassiope <> wrote:
    | [...] I want
    | to save a copy of the email in a particular directory which is
    | accessible to the Windows clients via samba.
    |
    | The strange thing is that even with the right user-id, I cannot seem
    | to write to the directory, getting an IOError exception. Changing the
    | directory to world-writable fixes this. I can confirm the uid and gid
    | for the script by having the script print these values just before
    | trying to create/write the file. Becoming the same lesser user, I
    | have no problem writing a file to the same directory.

    Can you show us:
    - the directory user and group ownership and permissions
    - the daemon's user and group values?

    You can also strace your daemon:

    strace -f -e trace=file your-daemon your-daemon-args... 2>strace.out

    and then examine the log for the precise UNIX-level failure.

    Cheers,
    --
    Cameron Simpson <> DoD#743
    http://www.cskk.ezoshosting.com/cs/

    Money won't buy happiness, but it will pay the salary of a large research
    staff to study the problem. - Bill Vaughan
    Cameron Simpson, Jan 3, 2010
    #3
  4. cassiope

    cassiope Guest

    On Jan 2, 3:46 pm, Steve Holden <> wrote:
    > cassiope wrote:
    > > I have a daemon on a Linux system that supports a number of Windows
    > > clients.  Among the functions is to send e-mails, which is
    > > sufficiently complicated that I fork() a separate process which gets
    > > setuid to a lesser user, and calls a python script which does the
    > > actual formatting and emailing (the daemon is written in C).  I want
    > > to save a copy of the email in a particular directory which is
    > > accessible to the Windows clients via samba.

    >
    > > The strange thing is that even with the right user-id, I cannot seem
    > > to write to the directory, getting an IOError exception.  Changing the
    > > directory to world-writable fixes this.  I can confirm the uid and gid
    > > for the script by having the script print these values just before
    > > trying to create/write the file.  Becoming the same lesser user, I
    > > have no problem writing a file to the same directory.

    >
    > Have you looked at the IOError's errno attribute to find out exactly why
    > the Python subprocess is unable to write to the directory?


    It's errno=13 ... "permission denied".

    > > Is there anything that I can do to diagnose why this script is
    > > failing?  For various reasons I don't want to make the directory world-
    > > writable.

    >
    > I'd concur on that decision.
    >
    > > This is on a Debian "squeeze" system, with python 2.5.

    >
    > > Thanks for any insights!

    >
    > Take a closer look at the exception, that might stimulate a thought or two.
    >
    > regards
    >  Steve
    > --
    > Steve Holden           +1 571 484 6266   +1 800 494 3119
    > PyCon is coming! Atlanta, Feb 2010  http://us.pycon.org/
    > Holden Web LLC                http://www.holdenweb.com/
    > UPCOMING EVENTS:        http://holdenweb.eventbrite.com/
    cassiope, Jan 3, 2010
    #4
  5. cassiope

    cassiope Guest

    On Jan 2, 6:40 pm, Christian Heimes <> wrote:
    > cassiope wrote:
    > > The strange thing is that even with the right user-id, I cannot seem
    > > to write to the directory, getting an IOError exception.  Changing the
    > > directory to world-writable fixes this.  I can confirm the uid and gid
    > > for the script by having the script print these values just before
    > > trying to create/write the file.  Becoming the same lesser user, I
    > > have no problem writing a file to the same directory.

    >
    > Are you able to write to the directory with the user id when you tried
    > to create a file manually?


    Yes. Sorry that wasn't clear.

    > How are you changing the uid and gid of your script? IIRC you have to
    > set the effective user id with os.seteuid() and os.setegid().


    I'm changing the uid and gid in the daemon (which runs with root
    permissions
    until the fork and uid/gid change). The uid and gid are confirmed by
    printing os.getuid() and os.getgid() in the script.

    > Christian
    cassiope, Jan 3, 2010
    #5
  6. cassiope

    Steve Holden Guest

    cassiope wrote:
    > On Jan 2, 6:40 pm, Christian Heimes <> wrote:
    >> cassiope wrote:
    >>> The strange thing is that even with the right user-id, I cannot seem
    >>> to write to the directory, getting an IOError exception. Changing the
    >>> directory to world-writable fixes this. I can confirm the uid and gid
    >>> for the script by having the script print these values just before
    >>> trying to create/write the file. Becoming the same lesser user, I
    >>> have no problem writing a file to the same directory.

    >> Are you able to write to the directory with the user id when you tried
    >> to create a file manually?

    >
    > Yes. Sorry that wasn't clear.
    >
    >> How are you changing the uid and gid of your script? IIRC you have to
    >> set the effective user id with os.seteuid() and os.setegid().

    >
    > I'm changing the uid and gid in the daemon (which runs with root
    > permissions
    > until the fork and uid/gid change). The uid and gid are confirmed by
    > printing os.getuid() and os.getgid() in the script.
    >

    And what do os.geteuid() and os.getegid() report? I suppose it's
    possible there's some bizarre difference between the effective and
    actual process parameters. IS the filestore a local file system, or an
    NFS mount?

    regards
    Steve
    --
    Steve Holden +1 571 484 6266 +1 800 494 3119
    PyCon is coming! Atlanta, Feb 2010 http://us.pycon.org/
    Holden Web LLC http://www.holdenweb.com/
    UPCOMING EVENTS: http://holdenweb.eventbrite.com/
    Steve Holden, Jan 3, 2010
    #6
  7. cassiope

    cassiope Guest

    On Jan 2, 8:02 pm, Cameron Simpson <> wrote:
    > On 02Jan2010 15:21, cassiope <> wrote:
    > | [...]  I want
    > | to save a copy of the email in a particular directory which is
    > | accessible to the Windows clients via samba.
    > |
    > | The strange thing is that even with the right user-id, I cannot seem
    > | to write to the directory, getting an IOError exception.  Changing the
    > | directory to world-writable fixes this.  I can confirm the uid and gid
    > | for the script by having the script print these values just before
    > | trying to create/write the file.  Becoming the same lesser user, I
    > | have no problem writing a file to the same directory.
    >
    > Can you show us:
    >   - the directory user and group ownership and permissions
    >   - the daemon's user and group values?


    Directory permissions: 774
    Directory ownership: "lesser user", "special group" where /etc/group
    has
    "special group" members including the "lesser user", as well as
    those
    who are expected to use the daemon, but not root.
    Script ownership: "lesser user"; permissions 755
    Daemon ownership: root; permissions: 755 (always started by root).

    The script also has to connect to a postgresql database for part of
    its
    work - that part works,
    > You can also strace your daemon:
    >
    >   strace -f -e trace=file your-daemon your-daemon-args... 2>strace.out
    >
    > and then examine the log for the precise UNIX-level failure.
    >
    > Cheers,
    > --
    > Cameron Simpson <> DoD#743http://www.cskk.ezoshosting.com/cs/
    >
    > Money won't buy happiness, but it will pay the salary of a large research
    > staff to study the problem. - Bill Vaughan


    Thanks, Cameron (and Steve and Christian). My first shot with strace
    (it's
    been awhile since I've used that - I think your syntax may be a tiny
    bit off
    - but it's probably the tool I need to use. Will explore further...
    cassiope, Jan 3, 2010
    #7
  8. On 03Jan2010 14:20, cassiope <> wrote:
    | On Jan 2, 8:02 pm, Cameron Simpson <> wrote:
    | > Can you show us:
    | >   - the directory user and group ownership and permissions
    | >   - the daemon's user and group values?
    |
    | Directory permissions: 774

    That's unusual - why the "4"? Directories with read but no search (1)
    are of limited use. (Not none - it's only unusual, not insane).

    | Directory ownership: "lesser user", "special group" where /etc/group
    | has "special group" members including the "lesser user", as well as
    | those who are expected to use the daemon, but not root.
    | Script ownership: "lesser user"; permissions 755
    | Daemon ownership: root; permissions: 755 (always started by root).

    And the script/daemon _runs_ as the "lesser user"?

    If so, superficially the permissions look like they should work.
    --
    Cameron Simpson <> DoD#743
    http://www.cskk.ezoshosting.com/cs/

    I couldn't think of anything else to do with it, so I put it on the web.
    Cameron Simpson, Jan 3, 2010
    #8
  9. cassiope

    cassiope Guest

    On Jan 3, 3:00 pm, Cameron Simpson <> wrote:
    > On 03Jan2010 14:20, cassiope <> wrote:
    > | On Jan 2, 8:02 pm, Cameron Simpson <> wrote:
    > | > Can you show us:
    > | >   - the directory user and group ownership and permissions
    > | >   - the daemon's user and group values?
    > |
    > | Directory permissions: 774
    >
    > That's unusual - why the "4"? Directories with read but no search (1)
    > are of limited use. (Not none - it's only unusual, not insane).
    >
    > | Directory ownership: "lesser user", "special group" where /etc/group
    > | has "special group" members including the "lesser user", as well as
    > | those who are expected to use the daemon, but not root.
    > | Script ownership: "lesser user"; permissions 755
    > | Daemon ownership: root; permissions: 755 (always started by root).
    >
    > And the script/daemon _runs_ as the "lesser user"?
    >
    > If so, superficially the permissions look like they should work.
    > --
    > Cameron Simpson <> DoD#743http://www.cskk.ezoshosting.com/cs/
    >
    > I couldn't think of anything else to do with it, so I put it on the web.


    Strace confirms the uid and gid == "lesser user". Changing the
    directory
    permissions to 775 changes nothing. Clearly get EACCES error on the
    attempted
    file creation.

    The only other thing is that as part of the python interpreter call, I
    provide
    a "reduced environment", just UID,GID,TMP,PWD,USER, and HOME. Is
    anything
    else needed?

    Thanks again, Cameron!
    cassiope, Jan 3, 2010
    #9
  10. On 03Jan2010 15:56, cassiope <> wrote:
    | Strace confirms the uid and gid == "lesser user". Changing the
    | directory
    | permissions to 775 changes nothing. Clearly get EACCES error on the
    | attempted
    | file creation.
    |
    | The only other thing is that as part of the python interpreter call, I
    | provide
    | a "reduced environment", just UID,GID,TMP,PWD,USER, and HOME. Is
    | anything
    | else needed?

    Should be irrelevant.

    Ok: does the file to be created already exist? If so, what are its
    permissions? If the file exists and isn't writable you may get this
    error.

    Also, did you eyeball the actual open() call to ensure the file pathname
    is correct, and doesn't use a bogus (non-existent) directory name?
    --
    Cameron Simpson <> DoD#743
    http://www.cskk.ezoshosting.com/cs/

    "GOD IS MY SOURCE" - Bumper sticker, Chapel Hill, NC
    I'll have to remember that one for the next code review meeting.
    - (Alain van der Heide)
    Cameron Simpson, Jan 4, 2010
    #10
  11. cassiope

    Nobody Guest

    On Sun, 03 Jan 2010 13:56:24 -0800, cassiope wrote:

    > I'm changing the uid and gid in the daemon (which runs with root
    > permissions
    > until the fork and uid/gid change). The uid and gid are confirmed by
    > printing os.getuid() and os.getgid() in the script.


    Those tell you the *real* UID/GID. Filesystem checks are performed using
    the *effective* UID/GID.

    You need to set them with seteuid/setegid and check them with
    geteuid/getegid.
    Nobody, Jan 4, 2010
    #11
  12. cassiope

    cassiope Guest

    On Jan 4, 7:46 am, Nobody <> wrote:
    > On Sun, 03 Jan 2010 13:56:24 -0800, cassiope wrote:
    > > I'm changing the uid and gid in the daemon (which runs with root
    > > permissions
    > > until the fork and uid/gid change).  The uid and gid are confirmed by
    > > printing os.getuid() and os.getgid() in the script.

    >
    > Those tell you the *real* UID/GID. Filesystem checks are performed using
    > the *effective* UID/GID.
    >
    > You need to set them with seteuid/setegid and check them with
    > geteuid/getegid.


    To Cameron: the file doesn't (yet) exist; and it has the correct full
    path.

    To "Nobody" : hey, this seems interesting. First test, invoking
    seteuid()
    and setegid() didn't help - but strange thing was these calls didn't
    show
    up in the strace, so perhaps I wasn't testing what I thought I was.
    I'll
    have to check this later as people are now back to work and needing
    the
    daemon for its other functions...

    Thanks for your ideas! I'll be checking it more later...
    cassiope, Jan 4, 2010
    #12
  13. On 04Jan2010 09:16, cassiope <> wrote:
    | To Cameron: the file doesn't (yet) exist; and it has the correct full
    | path.

    Can you show us the strace output of the failing open() call?

    | To "Nobody" : hey, this seems interesting. First test, invoking
    | seteuid()
    | and setegid() didn't help - but strange thing was these calls didn't
    | show
    | up in the strace, so perhaps I wasn't testing what I thought I was.

    If you're using the "-e trace=file" option it won't. That constrains the
    output to file operations to make the log easier to read. Discard the -e
    option to get everything.

    Cheers,
    --
    Cameron Simpson <> DoD#743
    http://www.cskk.ezoshosting.com/cs/

    Rimmer: It will be happened; it shall be going to be happening; it will be
    was an event that could will have been taken place in the future.
    - Red Dwarf, _Future Echoes_
    Cameron Simpson, Jan 5, 2010
    #13
  14. cassiope

    cassiope Guest

    On Jan 4, 4:23 pm, Cameron Simpson <> wrote:
    > On 04Jan2010 09:16, cassiope <> wrote:
    > | To Cameron: the file doesn't (yet) exist; and it has the correct full
    > | path.
    >
    > Can you show us the strace output of the failing open() call?


    Ah...presumably you mean:

    [pid 1976] open("/var/tmp/share/lvrq-Robert.Smith", O_WRONLY|
    O_CREAT|O_TRUNC, 0666) = -1 EACCES (Permission denied)

    > | To "Nobody" : hey, this seems interesting.  First test, invoking
    > | seteuid()
    > | and setegid() didn't help - but strange thing was these calls didn't
    > | show
    > | up in the strace, so perhaps I wasn't testing what I thought I was.
    >
    > If you're using the "-e trace=file" option it won't. That constrains the
    > output to file operations to make the log easier to read. Discard the -e
    > option to get everything.
    >
    > Cheers,
    > --
    > Cameron Simpson <> DoD#743http://www.cskk.ezoshosting.com/cs/


    Wasn't using the -e option. It turns out that the compiler was
    changing the
    code to use the linux functions setresgid() and setresuid(). That's
    why I
    didn't see it previously.

    If I only use the seteuid/setegid, it "works" - it is able to write
    the file.
    Unfortunately this may be partly due to its incompletely dropping
    priviledges -
    the file has root ownership, not "lesser user" ownership.

    Using setuid/setgid or setresuid/setresgid where just real OR both
    real and
    effective identities are set to "lesser user" - it still doesn't work
    - no file
    is written. Again, "lesser user" has no problem writing a file into
    this
    directory.

    I remain mystified :(
    Thanks for your valiant efforts!
    cassiope, Jan 5, 2010
    #14
  15. cassiope

    cassiope Guest

    One more tidbit observed: my last note, that it works when using
    seteuid/setegid?
    Well - that only applies if the daemon is running under strace (!).
    It fails
    if started directly by root, or if the strace session has ended,
    leaving the
    main body of the daemon running in its normal headless manner.

    I wonder if running under "strace -f" - might setegid/seteuid be
    prevented from
    having their normal effect?

    Sigh.
    cassiope, Jan 5, 2010
    #15
  16. cassiope

    Nobody Guest

    On Mon, 04 Jan 2010 21:30:31 -0800, cassiope wrote:

    > One more tidbit observed: my last note, that it works when using
    > seteuid/setegid?
    > Well - that only applies if the daemon is running under strace (!).
    > It fails
    > if started directly by root, or if the strace session has ended,
    > leaving the
    > main body of the daemon running in its normal headless manner.
    >
    > I wonder if running under "strace -f" - might setegid/seteuid be
    > prevented from
    > having their normal effect?


    Possibly. The ptrace() syscall on which strace depends will fail if you
    try to trace a "privileged" process and you aren't root, so it's possible
    that a ptrace()d process will refuse to become privileged.

    Here, "privileged" includes a process which has changed any of its UIDs or
    GIDs (this prevents a normal user from tracing, killing, etc an otherwise
    privileged process which has switched to the user's UID for the time being).
    Nobody, Jan 5, 2010
    #16
  17. cassiope

    cassiope Guest

    On Jan 5, 10:58 am, Nobody <> wrote:
    > On Mon, 04 Jan 2010 21:30:31 -0800, cassiope wrote:
    > > One more tidbit observed: my last note, that it works when using
    > > seteuid/setegid?
    > > Well - that only applies if the daemon is running under strace (!).
    > > It fails
    > > if started directly by root, or if the strace session has ended,
    > > leaving the
    > > main body of the daemon running in its normal headless manner.

    >
    > > I wonder if running under "strace -f" - might setegid/seteuid be
    > > prevented from
    > > having their normal effect?

    >
    > Possibly. The ptrace() syscall on which strace depends will fail if you
    > try to trace a "privileged" process and you aren't root, so it's possible
    > that a ptrace()d process will refuse to become privileged.
    >
    > Here, "privileged" includes a process which has changed any of its UIDs or
    > GIDs (this prevents a normal user from tracing, killing, etc an otherwise
    > privileged process which has switched to the user's UID for the time being).


    Thanks.

    I guess it's time to trim this beast to a simpler state to isolate
    what's
    going wrong. This may take awhile amidst everything else I have to be
    doing...
    cassiope, Jan 6, 2010
    #17
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    2
    Views:
    1,646
    Fredrik Lundh
    Aug 20, 2006
  2. Jean-Paul Calderone
    Replies:
    0
    Views:
    455
    Jean-Paul Calderone
    Mar 20, 2009
  3. Floris Bruynooghe
    Replies:
    1
    Views:
    451
    Floris Bruynooghe
    Mar 24, 2009
  4. volcano
    Replies:
    4
    Views:
    1,777
    volcano
    Sep 24, 2009
  5. Daemon Win32::Daemon;

    , Sep 7, 2006, in forum: Perl Misc
    Replies:
    0
    Views:
    251
Loading...

Share This Page