Daemon loses __file__ reference after a while.

Discussion in 'Python' started by ivdneut@gmail.com, Jul 24, 2012.

  1. Guest

    Hello,

    I have a daemon process that runs for a considerable amount of time (weeks on end) without any problems. At some point I start getting the exception:

    Exception info: Traceback (most recent call last):
    File "scheduler.py", line 376, in applyrule
    result = execrule(rule_code)
    File "scheduler.py", line 521, in execrule
    rulepath = os.path.dirname(__file__)+"/"+'/'.join(rule['modules'])+"/"+rule['rulename']
    NameError: name '__file__' is not defined

    This section of the code is executed in this process *all the time*, but suddenly stops working. I have been searching for similar issues online, but only come accross people having problems because they run the script interactively. This is not the case here.

    I am running python from a virtual-env installation from a stock Red Hat EL6.2 installation:

    (virtual-env)[user@host ~]$ python --version
    Python 2.6.6
    (virtual-env)[user@host ~]$ cat /etc/redhat-release
    Red Hat Enterprise Linux Server release 6.2 (Santiago)

    I would greatly appreciate any pointers on where to start looking to find the problem.

    Ian.
    , Jul 24, 2012
    #1
    1. Advertising

  2. hello,

    On Tue, Jul 24, 2012 at 04:48:42AM -0700, wrote:
    > Hello,
    >
    > I have a daemon process that runs for a considerable amount of time (weeks on end) without any problems. At some point I start getting the exception:
    >
    > Exception info: Traceback (most recent call last):
    > File "scheduler.py", line 376, in applyrule
    > result = execrule(rule_code)
    > File "scheduler.py", line 521, in execrule
    > rulepath = os.path.dirname(__file__)+"/"+'/'.join(rule['modules'])+"/"+rule['rulename']
    > NameError: name '__file__' is not defined
    >
    > This section of the code is executed in this process *all the time*, but suddenly stops working. I have been searching for similar issues online, but only come accross people having problems because they run the script interactively. This is not the case here.


    could you send the relevant part of the code?

    I mean: how do you daemonize your process?

    > I am running python from a virtual-env installation from a stock Red Hat EL 6.2 installation:
    >
    > (virtual-env)[user@host ~]$ python --version
    > Python 2.6.6
    > (virtual-env)[user@host ~]$ cat /etc/redhat-release
    > Red Hat Enterprise Linux Server release 6.2 (Santiago)


    If you use fork(), it drops all file descriptors, and creates new
    ones - may be then loss the __file__...?


    a.


    --
    I � UTF-8
    Ervin Hegedüs, Jul 24, 2012
    #2
    1. Advertising

  3. Laszlo Nagy Guest


    > If you use fork(), it drops all file descriptors, and creates new
    > ones - may be then loss the __file__...?
    >

    I don't think this is the case. He wrote that the process runs for weeks
    without problems, and code using __file__ is being executed all the time.
    Laszlo Nagy, Jul 24, 2012
    #3
  4. Guest

    On Tuesday, July 24, 2012 2:24:31 PM UTC+2, Ervin Hegedüs wrote:
    > hello,
    >
    > On Tue, Jul 24, 2012 at 04:48:42AM -0700, wrote:
    > > Hello,
    > >
    > > I have a daemon process that runs for a considerable amount of time (weeks on end) without any problems. At some point I start getting the exception:
    > >
    > > Exception info: Traceback (most recent call last):
    > > File "scheduler.py", line 376, in applyrule
    > > result = execrule(rule_code)
    > > File "scheduler.py", line 521, in execrule
    > > rulepath = os.path.dirname(__file__)+"/"+'/'..join(rule['modules'])+"/"+rule['rulename']
    > > NameError: name '__file__' is not defined
    > >
    > > This section of the code is executed in this process *all the time*,but suddenly stops working. I have been searching for similar issues online, but only come accross people having problems because they run the scriptinteractively. This is not the case here.
    >
    > could you send the relevant part of the code?
    >
    > I mean: how do you daemonize your process?


    It's done by a double fork:

    ## First fork()
    pid = os.fork()
    if pid != 0: sys.exit(0) # parent exits.

    ## create new session
    os.setsid()

    ## ignore SIGHUP
    signal.signal(signal.SIGHUP, signal.SIG_IGN)

    ## Second fork()
    pid = os.fork()
    if pid != 0: sys.exit(0) # First child exits.

    ## Change working directory to the home directory.
    homedir = pwd.getpwuid(os.geteuid())[5]
    os.chdir(homedir)

    os.umask(0)

    for fd in range(0, 1024):
    try:
    os.close(fd)
    except:
    pass # fd not open, ignore this exception.

    The original C version of this code is from W.R. Stevens' daemon_init() routine in "UNIX Network Programming Volume 1, second edition"

    >
    > > I am running python from a virtual-env installation from a stock RedHat EL 6.2 installation:
    > >
    > > (virtual-env)[user@host ~]$ python --version
    > > Python 2.6.6
    > > (virtual-env)[user@host ~]$ cat /etc/redhat-release
    > > Red Hat Enterprise Linux Server release 6.2 (Santiago)
    >
    > If you use fork(), it drops all file descriptors, and creates new
    > ones - may be then loss the __file__...?


    I doubt this would be it, or it would stop working immediately, since daemonization is done upon startup of the process. File descriptors are closed immediately upon startup, it doesn't seem to affect the reference to the __file__ string (which is not a file object, but a str object)

    >
    >
    > a.
    >
    >
    > --
    > I � UTF-8




    On Tuesday, July 24, 2012 2:24:31 PM UTC+2, Ervin Hegedüs wrote:
    > hello,
    >
    > On Tue, Jul 24, 2012 at 04:48:42AM -0700, wrote:
    > > Hello,
    > >
    > > I have a daemon process that runs for a considerable amount of time (weeks on end) without any problems. At some point I start getting the exception:
    > >
    > > Exception info: Traceback (most recent call last):
    > > File "scheduler.py", line 376, in applyrule
    > > result = execrule(rule_code)
    > > File "scheduler.py", line 521, in execrule
    > > rulepath = os.path.dirname(__file__)+"/"+'/'..join(rule['modules'])+"/"+rule['rulename']
    > > NameError: name '__file__' is not defined
    > >
    > > This section of the code is executed in this process *all the time*,but suddenly stops working. I have been searching for similar issues online, but only come accross people having problems because they run the scriptinteractively. This is not the case here.
    >
    > could you send the relevant part of the code?
    >
    > I mean: how do you daemonize your process?
    >
    > > I am running python from a virtual-env installation from a stock RedHat EL 6.2 installation:
    > >
    > > (virtual-env)[user@host ~]$ python --version
    > > Python 2.6.6
    > > (virtual-env)[user@host ~]$ cat /etc/redhat-release
    > > Red Hat Enterprise Linux Server release 6.2 (Santiago)
    >
    > If you use fork(), it drops all file descriptors, and creates new
    > ones - may be then loss the __file__...?
    >
    >
    > a.
    >
    >
    > --
    > I � UTF-8
    , Jul 24, 2012
    #4
  5. Guest

    On Tuesday, July 24, 2012 2:24:31 PM UTC+2, Ervin Hegedüs wrote:
    > hello,
    >
    > On Tue, Jul 24, 2012 at 04:48:42AM -0700, wrote:
    > > Hello,
    > >
    > > I have a daemon process that runs for a considerable amount of time (weeks on end) without any problems. At some point I start getting the exception:
    > >
    > > Exception info: Traceback (most recent call last):
    > > File "scheduler.py", line 376, in applyrule
    > > result = execrule(rule_code)
    > > File "scheduler.py", line 521, in execrule
    > > rulepath = os.path.dirname(__file__)+"/"+'/'..join(rule['modules'])+"/"+rule['rulename']
    > > NameError: name '__file__' is not defined
    > >
    > > This section of the code is executed in this process *all the time*,but suddenly stops working. I have been searching for similar issues online, but only come accross people having problems because they run the scriptinteractively. This is not the case here.
    >
    > could you send the relevant part of the code?
    >
    > I mean: how do you daemonize your process?


    It's done by a double fork:

    ## First fork()
    pid = os.fork()
    if pid != 0: sys.exit(0) # parent exits.

    ## create new session
    os.setsid()

    ## ignore SIGHUP
    signal.signal(signal.SIGHUP, signal.SIG_IGN)

    ## Second fork()
    pid = os.fork()
    if pid != 0: sys.exit(0) # First child exits.

    ## Change working directory to the home directory.
    homedir = pwd.getpwuid(os.geteuid())[5]
    os.chdir(homedir)

    os.umask(0)

    for fd in range(0, 1024):
    try:
    os.close(fd)
    except:
    pass # fd not open, ignore this exception.

    The original C version of this code is from W.R. Stevens' daemon_init() routine in "UNIX Network Programming Volume 1, second edition"

    >
    > > I am running python from a virtual-env installation from a stock RedHat EL 6.2 installation:
    > >
    > > (virtual-env)[user@host ~]$ python --version
    > > Python 2.6.6
    > > (virtual-env)[user@host ~]$ cat /etc/redhat-release
    > > Red Hat Enterprise Linux Server release 6.2 (Santiago)
    >
    > If you use fork(), it drops all file descriptors, and creates new
    > ones - may be then loss the __file__...?


    I doubt this would be it, or it would stop working immediately, since daemonization is done upon startup of the process. File descriptors are closed immediately upon startup, it doesn't seem to affect the reference to the __file__ string (which is not a file object, but a str object)

    >
    >
    > a.
    >
    >
    > --
    > I � UTF-8




    On Tuesday, July 24, 2012 2:24:31 PM UTC+2, Ervin Hegedüs wrote:
    > hello,
    >
    > On Tue, Jul 24, 2012 at 04:48:42AM -0700, wrote:
    > > Hello,
    > >
    > > I have a daemon process that runs for a considerable amount of time (weeks on end) without any problems. At some point I start getting the exception:
    > >
    > > Exception info: Traceback (most recent call last):
    > > File "scheduler.py", line 376, in applyrule
    > > result = execrule(rule_code)
    > > File "scheduler.py", line 521, in execrule
    > > rulepath = os.path.dirname(__file__)+"/"+'/'..join(rule['modules'])+"/"+rule['rulename']
    > > NameError: name '__file__' is not defined
    > >
    > > This section of the code is executed in this process *all the time*,but suddenly stops working. I have been searching for similar issues online, but only come accross people having problems because they run the scriptinteractively. This is not the case here.
    >
    > could you send the relevant part of the code?
    >
    > I mean: how do you daemonize your process?
    >
    > > I am running python from a virtual-env installation from a stock RedHat EL 6.2 installation:
    > >
    > > (virtual-env)[user@host ~]$ python --version
    > > Python 2.6.6
    > > (virtual-env)[user@host ~]$ cat /etc/redhat-release
    > > Red Hat Enterprise Linux Server release 6.2 (Santiago)
    >
    > If you use fork(), it drops all file descriptors, and creates new
    > ones - may be then loss the __file__...?
    >
    >
    > a.
    >
    >
    > --
    > I � UTF-8
    , Jul 24, 2012
    #5
  6. "" <> writes:

    > I have a daemon process that runs for a considerable amount of time (weeks on end) without any problems. At some point I start getting the exception:
    >
    > Exception info: Traceback (most recent call last):
    > File "scheduler.py", line 376, in applyrule
    > result = execrule(rule_code)
    > File "scheduler.py", line 521, in execrule
    > rulepath = os.path.dirname(__file__)+"/"+'/'.join(rule['modules'])+"/"+rule['rulename']
    > NameError: name '__file__' is not defined
    >
    > This section of the code is executed in this process *all the time*, but suddenly stops working. I have been searching for similar issues online, but only come accross people having problems because they run the script interactively. This is not the case here.


    This is strange indeed.

    I have only one vague idea: should something try to terminate the
    process, modules would start to lose their variables during shutdown.
    Dieter Maurer, Jul 24, 2012
    #6
  7. Paul Rubin Guest

    Dieter Maurer <> writes:
    > I have only one vague idea: should something try to terminate the
    > process, modules would start to lose their variables during shutdown.


    That happens all the time with multi-threaded programs, because the
    shutdown is happening concurrently with other threads doing stuff. Are
    there threads in this particular program?
    Paul Rubin, Jul 24, 2012
    #7
  8. Ian Kelly Guest

    On Tue, Jul 24, 2012 at 8:59 AM, Dieter Maurer <> wrote:
    > "" <> writes:
    >
    >> I have a daemon process that runs for a considerable amount of time (weeks on end) without any problems. At some point I start getting the exception:
    >>
    >> Exception info: Traceback (most recent call last):
    >> File "scheduler.py", line 376, in applyrule
    >> result = execrule(rule_code)
    >> File "scheduler.py", line 521, in execrule
    >> rulepath = os.path.dirname(__file__)+"/"+'/'.join(rule['modules'])+"/"+rule['rulename']
    >> NameError: name '__file__' is not defined
    >>
    >> This section of the code is executed in this process *all the time*, butsuddenly stops working. I have been searching for similar issues online, but only come accross people having problems because they run the script interactively. This is not the case here.

    >
    > This is strange indeed.
    >
    > I have only one vague idea: should something try to terminate the
    > process, modules would start to lose their variables during shutdown.


    That's a good theory. Or perhaps something in the code itself is
    handling the module's globals() and very occasionally does something
    that is incorrect and destructive.
    Ian Kelly, Jul 24, 2012
    #8
  9. Ian Kelly Guest

    On Tue, Jul 24, 2012 at 1:32 PM, Paul Rubin <> wrote:
    > Dieter Maurer <> writes:
    >> I have only one vague idea: should something try to terminate the
    >> process, modules would start to lose their variables during shutdown.

    >
    > That happens all the time with multi-threaded programs, because the
    > shutdown is happening concurrently with other threads doing stuff. Are
    > there threads in this particular program?


    It also comes up in single-threaded programs that use finalizers
    (__del__ methods). At the time an object is finalized, many globals
    might already be gone.
    Ian Kelly, Jul 24, 2012
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. klenwell

    __file__ vs __FILE__

    klenwell, Nov 3, 2007, in forum: Python
    Replies:
    11
    Views:
    896
    Matimus
    Nov 5, 2007
  2. Jean-Paul Calderone
    Replies:
    0
    Views:
    441
    Jean-Paul Calderone
    Mar 20, 2009
  3. Floris Bruynooghe
    Replies:
    1
    Views:
    437
    Floris Bruynooghe
    Mar 24, 2009
  4. Daemon Win32::Daemon;

    , Sep 7, 2006, in forum: Perl Misc
    Replies:
    0
    Views:
    233
  5. Replies:
    3
    Views:
    230
Loading...

Share This Page