Daemon loses __file__ reference after a while.

[ivdneut]

Hello,

I have a daemon process that runs for a considerable amount of time (weeks on end) without any problems. At some point I start getting the exception:

Exception info: Traceback (most recent call last):
File "scheduler.py", line 376, in applyrule
result = execrule(rule_code)
File "scheduler.py", line 521, in execrule
rulepath = os.path.dirname(__file__)+"/"+'/'.join(rule['modules'])+"/"+rule['rulename']
NameError: name '__file__' is not defined

This section of the code is executed in this process *all the time*, but suddenly stops working. I have been searching for similar issues online, but only come accross people having problems because they run the script interactively. This is not the case here.

I am running python from a virtual-env installation from a stock Red Hat EL6.2 installation:

(virtual-env)[user@host ~]$ python --version
Python 2.6.6
(virtual-env)[user@host ~]$ cat /etc/redhat-release
Red Hat Enterprise Linux Server release 6.2 (Santiago)

I would greatly appreciate any pointers on where to start looking to find the problem.

Ian.

 

[Ervin Hegedüs]

hello,

Hello,

I have a daemon process that runs for a considerable amount of time (weeks on end) without any problems. At some point I start getting the exception:

Exception info: Traceback (most recent call last):
File "scheduler.py", line 376, in applyrule
result = execrule(rule_code)
File "scheduler.py", line 521, in execrule
rulepath = os.path.dirname(__file__)+"/"+'/'.join(rule['modules'])+"/"+rule['rulename']
NameError: name '__file__' is not defined

This section of the code is executed in this process *all the time*, but suddenly stops working. I have been searching for similar issues online, but only come accross people having problems because they run the script interactively. This is not the case here.

could you send the relevant part of the code?

I mean: how do you daemonize your process?

I am running python from a virtual-env installation from a stock Red Hat EL 6.2 installation:

(virtual-env)[user@host ~]$ python --version
Python 2.6.6
(virtual-env)[user@host ~]$ cat /etc/redhat-release
Red Hat Enterprise Linux Server release 6.2 (Santiago)

If you use fork(), it drops all file descriptors, and creates new
ones - may be then loss the __file__...?


a.

 

[Laszlo Nagy]

If you use fork(), it drops all file descriptors, and creates new
ones - may be then loss the __file__...?

I don't think this is the case. He wrote that the process runs for weeks
without problems, and code using __file__ is being executed all the time.

 

[ivdneut]

hello,

On Tue, Jul 24, 2012 at 04:48:42AM -0700, (e-mail address removed) wrote:
> Hello,
>
> I have a daemon process that runs for a considerable amount of time (weeks on end) without any problems. At some point I start getting the exception:
>
> Exception info: Traceback (most recent call last):
> File "scheduler.py", line 376, in applyrule
> result = execrule(rule_code)
> File "scheduler.py", line 521, in execrule
> rulepath = os.path.dirname(__file__)+"/"+'/'..join(rule['modules'])+"/"+rule['rulename']
> NameError: name '__file__' is not defined
>
> This section of the code is executed in this process *all the time*,but suddenly stops working. I have been searching for similar issues online, but only come accross people having problems because they run the scriptinteractively. This is not the case here.

could you send the relevant part of the code?

I mean: how do you daemonize your process?

It's done by a double fork:

## First fork()
pid = os.fork()
if pid != 0: sys.exit(0) # parent exits.

## create new session
os.setsid()

## ignore SIGHUP
signal.signal(signal.SIGHUP, signal.SIG_IGN)

## Second fork()
pid = os.fork()
if pid != 0: sys.exit(0) # First child exits.

## Change working directory to the home directory.
homedir = pwd.getpwuid(os.geteuid())[5]
os.chdir(homedir)

os.umask(0)

for fd in range(0, 1024):
try:
os.close(fd)
except:
pass # fd not open, ignore this exception.

The original C version of this code is from W.R. Stevens' daemon_init() routine in "UNIX Network Programming Volume 1, second edition"

> I am running python from a virtual-env installation from a stock RedHat EL 6.2 installation:
>
> (virtual-env)[user@host ~]$ python --version
> Python 2.6.6
> (virtual-env)[user@host ~]$ cat /etc/redhat-release
> Red Hat Enterprise Linux Server release 6.2 (Santiago)

If you use fork(), it drops all file descriptors, and creates new
ones - may be then loss the __file__...?

I doubt this would be it, or it would stop working immediately, since daemonization is done upon startup of the process. File descriptors are closed immediately upon startup, it doesn't seem to affect the reference to the __file__ string (which is not a file object, but a str object)

a.


--
I � UTF-8



hello,

On Tue, Jul 24, 2012 at 04:48:42AM -0700, (e-mail address removed) wrote:
> Hello,
>
> I have a daemon process that runs for a considerable amount of time (weeks on end) without any problems. At some point I start getting the exception:
>
> Exception info: Traceback (most recent call last):
> File "scheduler.py", line 376, in applyrule
> result = execrule(rule_code)
> File "scheduler.py", line 521, in execrule
> rulepath = os.path.dirname(__file__)+"/"+'/'..join(rule['modules'])+"/"+rule['rulename']
> NameError: name '__file__' is not defined
>
> This section of the code is executed in this process *all the time*,but suddenly stops working. I have been searching for similar issues online, but only come accross people having problems because they run the scriptinteractively. This is not the case here.

could you send the relevant part of the code?

I mean: how do you daemonize your process?

> I am running python from a virtual-env installation from a stock RedHat EL 6.2 installation:
>
> (virtual-env)[user@host ~]$ python --version
> Python 2.6.6
> (virtual-env)[user@host ~]$ cat /etc/redhat-release
> Red Hat Enterprise Linux Server release 6.2 (Santiago)

If you use fork(), it drops all file descriptors, and creates new
ones - may be then loss the __file__...?


a.

 

[ivdneut]

hello,

On Tue, Jul 24, 2012 at 04:48:42AM -0700, (e-mail address removed) wrote:
> Hello,
>
> I have a daemon process that runs for a considerable amount of time (weeks on end) without any problems. At some point I start getting the exception:
>
> Exception info: Traceback (most recent call last):
> File "scheduler.py", line 376, in applyrule
> result = execrule(rule_code)
> File "scheduler.py", line 521, in execrule
> rulepath = os.path.dirname(__file__)+"/"+'/'..join(rule['modules'])+"/"+rule['rulename']
> NameError: name '__file__' is not defined
>
> This section of the code is executed in this process *all the time*,but suddenly stops working. I have been searching for similar issues online, but only come accross people having problems because they run the scriptinteractively. This is not the case here.

could you send the relevant part of the code?

I mean: how do you daemonize your process?

It's done by a double fork:

## First fork()
pid = os.fork()
if pid != 0: sys.exit(0) # parent exits.

## create new session
os.setsid()

## ignore SIGHUP
signal.signal(signal.SIGHUP, signal.SIG_IGN)

## Second fork()
pid = os.fork()
if pid != 0: sys.exit(0) # First child exits.

## Change working directory to the home directory.
homedir = pwd.getpwuid(os.geteuid())[5]
os.chdir(homedir)

os.umask(0)

for fd in range(0, 1024):
try:
os.close(fd)
except:
pass # fd not open, ignore this exception.

The original C version of this code is from W.R. Stevens' daemon_init() routine in "UNIX Network Programming Volume 1, second edition"

> I am running python from a virtual-env installation from a stock RedHat EL 6.2 installation:
>
> (virtual-env)[user@host ~]$ python --version
> Python 2.6.6
> (virtual-env)[user@host ~]$ cat /etc/redhat-release
> Red Hat Enterprise Linux Server release 6.2 (Santiago)

If you use fork(), it drops all file descriptors, and creates new
ones - may be then loss the __file__...?

I doubt this would be it, or it would stop working immediately, since daemonization is done upon startup of the process. File descriptors are closed immediately upon startup, it doesn't seem to affect the reference to the __file__ string (which is not a file object, but a str object)

a.


--
I � UTF-8



hello,

On Tue, Jul 24, 2012 at 04:48:42AM -0700, (e-mail address removed) wrote:
> Hello,
>
> I have a daemon process that runs for a considerable amount of time (weeks on end) without any problems. At some point I start getting the exception:
>
> Exception info: Traceback (most recent call last):
> File "scheduler.py", line 376, in applyrule
> result = execrule(rule_code)
> File "scheduler.py", line 521, in execrule
> rulepath = os.path.dirname(__file__)+"/"+'/'..join(rule['modules'])+"/"+rule['rulename']
> NameError: name '__file__' is not defined
>
> This section of the code is executed in this process *all the time*,but suddenly stops working. I have been searching for similar issues online, but only come accross people having problems because they run the scriptinteractively. This is not the case here.

could you send the relevant part of the code?

I mean: how do you daemonize your process?

> I am running python from a virtual-env installation from a stock RedHat EL 6.2 installation:
>
> (virtual-env)[user@host ~]$ python --version
> Python 2.6.6
> (virtual-env)[user@host ~]$ cat /etc/redhat-release
> Red Hat Enterprise Linux Server release 6.2 (Santiago)

If you use fork(), it drops all file descriptors, and creates new
ones - may be then loss the __file__...?


a.

 

[Dieter Maurer]

I have a daemon process that runs for a considerable amount of time (weeks on end) without any problems. At some point I start getting the exception:

Exception info: Traceback (most recent call last):
File "scheduler.py", line 376, in applyrule
result = execrule(rule_code)
File "scheduler.py", line 521, in execrule
rulepath = os.path.dirname(__file__)+"/"+'/'.join(rule['modules'])+"/"+rule['rulename']
NameError: name '__file__' is not defined

This section of the code is executed in this process *all the time*, but suddenly stops working. I have been searching for similar issues online, but only come accross people having problems because they run the script interactively. This is not the case here.

This is strange indeed.

I have only one vague idea: should something try to terminate the
process, modules would start to lose their variables during shutdown.

 

[Paul Rubin]

I have only one vague idea: should something try to terminate the
process, modules would start to lose their variables during shutdown.

That happens all the time with multi-threaded programs, because the
shutdown is happening concurrently with other threads doing stuff. Are
there threads in this particular program?

 

[Ian Kelly]

I have a daemon process that runs for a considerable amount of time (weeks on end) without any problems. At some point I start getting the exception:

Exception info: Traceback (most recent call last):
File "scheduler.py", line 376, in applyrule
result = execrule(rule_code)
File "scheduler.py", line 521, in execrule
rulepath = os.path.dirname(__file__)+"/"+'/'.join(rule['modules'])+"/"+rule['rulename']
NameError: name '__file__' is not defined

This section of the code is executed in this process *all the time*, butsuddenly stops working. I have been searching for similar issues online, but only come accross people having problems because they run the script interactively. This is not the case here.

This is strange indeed.

I have only one vague idea: should something try to terminate the
process, modules would start to lose their variables during shutdown.

That's a good theory. Or perhaps something in the code itself is
handling the module's globals() and very occasionally does something
that is incorrect and destructive.

 

[Ian Kelly]

That happens all the time with multi-threaded programs, because the
shutdown is happening concurrently with other threads doing stuff. Are
there threads in this particular program?

It also comes up in single-threaded programs that use finalizers
(__del__ methods). At the time an object is finalized, many globals
might already be gone.