Only Bytecode, No .py Files

[Eldon Ziegler]

Is there a way to have the Python processor look only for bytecode
files, not .py files? We are seeing huge numbers of Linux audit messages
on production system on which only bytecode files are stored. The audit
subsystem is recording each open failure.

Thanks,
Eldon Ziegler

 

[Billy Mays]

Is there a way to have the Python processor look only for bytecode
files, not .py files? We are seeing huge numbers of Linux audit messages
on production system on which only bytecode files are stored. The audit
subsystem is recording each open failure.

Thanks,
Eldon Ziegler

How are you opening your files?

 

[Thomas Rachel]

Am 26.07.2011 17:19 schrieb Eldon Ziegler:

Is there a way to have the Python processor look only for bytecode
files, not .py files? We are seeing huge numbers of Linux audit messages
on production system on which only bytecode files are stored. The audit
subsystem is recording each open failure.

Is that really a problem? AFAIK there are many failing open() calls on
the start of every program.

E.g.

open("/lib/bash/4.1/tls/i686/sse2/libncurses.so.5", O_RDONLY) = -1
ENOENT (No such file or directory)
stat64("/lib/bash/4.1/tls/i686/sse2", 0xbfd3a350) = -1 ENOENT (No such
file or directory)
open("/lib/bash/4.1/tls/i686/libncurses.so.5", O_RDONLY) = -1 ENOENT (No
such file or directory)
stat64("/lib/bash/4.1/tls/i686", 0xbfd3a350) = -1 ENOENT (No such file
or directory)
open("/lib/bash/4.1/tls/sse2/libncurses.so.5", O_RDONLY) = -1 ENOENT (No
such file or directory)
stat64("/lib/bash/4.1/tls/sse2", 0xbfd3a350) = -1 ENOENT (No such file
or directory)
open("/lib/bash/4.1/tls/libncurses.so.5", O_RDONLY) = -1 ENOENT (No such
file or directory)
stat64("/lib/bash/4.1/tls", 0xbfd3a350) = -1 ENOENT (No such file or
directory)
open("/lib/bash/4.1/i686/sse2/libncurses.so.5", O_RDONLY) = -1 ENOENT
(No such file or directory)
stat64("/lib/bash/4.1/i686/sse2", 0xbfd3a350) = -1 ENOENT (No such file
or directory)
open("/lib/bash/4.1/i686/libncurses.so.5", O_RDONLY) = -1 ENOENT (No
such file or directory)
stat64("/lib/bash/4.1/i686", 0xbfd3a350) = -1 ENOENT (No such file or
directory)
open("/lib/bash/4.1/sse2/libncurses.so.5", O_RDONLY) = -1 ENOENT (No
such file or directory)
stat64("/lib/bash/4.1/sse2", 0xbfd3a350) = -1 ENOENT (No such file or
directory)
open("/lib/bash/4.1/libncurses.so.5", O_RDONLY) = -1 ENOENT (No such
file or directory)
stat64("/lib/bash/4.1", 0xbfd3a350) = -1 ENOENT (No such file or
directory)
open("/lib/libncurses.so.5", O_RDONLY) = 3

is a part of what happens if I start the MySQL client,

Even starting the bash results in

open("/lib/bash/4.1/tls/i686/sse2/libreadline.so.6", O_RDONLY) = -1
ENOENT (No such file or directory)
stat64("/lib/bash/4.1/tls/i686/sse2", 0xbfe0c4d0) = -1 ENOENT (No such
file or directory)
open("/lib/bash/4.1/tls/i686/libreadline.so.6", O_RDONLY) = -1 ENOENT
(No such file or directory)
stat64("/lib/bash/4.1/tls/i686", 0xbfe0c4d0) = -1 ENOENT (No such file
or directory)
open("/lib/bash/4.1/tls/sse2/libreadline.so.6", O_RDONLY) = -1 ENOENT
(No such file or directory)
stat64("/lib/bash/4.1/tls/sse2", 0xbfe0c4d0) = -1 ENOENT (No such file
or directory)
open("/lib/bash/4.1/tls/libreadline.so.6", O_RDONLY) = -1 ENOENT (No
such file or directory)
stat64("/lib/bash/4.1/tls", 0xbfe0c4d0) = -1 ENOENT (No such file or
directory)
open("/lib/bash/4.1/i686/sse2/libreadline.so.6", O_RDONLY) = -1 ENOENT
(No such file or directory)
stat64("/lib/bash/4.1/i686/sse2", 0xbfe0c4d0) = -1 ENOENT (No such file
or directory)
open("/lib/bash/4.1/i686/libreadline.so.6", O_RDONLY) = -1 ENOENT (No
such file or directory)
stat64("/lib/bash/4.1/i686", 0xbfe0c4d0) = -1 ENOENT (No such file or
directory)
open("/lib/bash/4.1/sse2/libreadline.so.6", O_RDONLY) = -1 ENOENT (No
such file or directory)
stat64("/lib/bash/4.1/sse2", 0xbfe0c4d0) = -1 ENOENT (No such file or
directory)
open("/lib/bash/4.1/libreadline.so.6", O_RDONLY) = -1 ENOENT (No such
file or directory)
stat64("/lib/bash/4.1", 0xbfe0c4d0) = -1 ENOENT (No such file or
directory)
open("/etc/ld.so.cache", O_RDONLY) = 3

So can it really be such a huge problem?


Thomas

 

[John Roth]

Am 26.07.2011 17:19 schrieb Eldon Ziegler:


Is that really a problem? AFAIK there are many failing open() calls on
the start of every program.

E.g.

open("/lib/bash/4.1/tls/i686/sse2/libncurses.so.5", O_RDONLY) = -1
ENOENT (No such file or directory)
stat64("/lib/bash/4.1/tls/i686/sse2", 0xbfd3a350) = -1 ENOENT (No such
file or directory)
open("/lib/bash/4.1/tls/i686/libncurses.so.5", O_RDONLY) = -1 ENOENT (No
such file or directory)
stat64("/lib/bash/4.1/tls/i686", 0xbfd3a350) = -1 ENOENT (No such file
or directory)
open("/lib/bash/4.1/tls/sse2/libncurses.so.5", O_RDONLY) = -1 ENOENT (No
such file or directory)
stat64("/lib/bash/4.1/tls/sse2", 0xbfd3a350) = -1 ENOENT (No such file
or directory)
open("/lib/bash/4.1/tls/libncurses.so.5", O_RDONLY) = -1 ENOENT (No such
file or directory)
stat64("/lib/bash/4.1/tls", 0xbfd3a350) = -1 ENOENT (No such file or
directory)
open("/lib/bash/4.1/i686/sse2/libncurses.so.5", O_RDONLY) = -1 ENOENT
(No such file or directory)
stat64("/lib/bash/4.1/i686/sse2", 0xbfd3a350) = -1 ENOENT (No such file
or directory)
open("/lib/bash/4.1/i686/libncurses.so.5", O_RDONLY) = -1 ENOENT (No
such file or directory)
stat64("/lib/bash/4.1/i686", 0xbfd3a350) = -1 ENOENT (No such file or
directory)
open("/lib/bash/4.1/sse2/libncurses.so.5", O_RDONLY) = -1 ENOENT (No
such file or directory)
stat64("/lib/bash/4.1/sse2", 0xbfd3a350) = -1 ENOENT (No such file or
directory)
open("/lib/bash/4.1/libncurses.so.5", O_RDONLY) = -1 ENOENT (No such
file or directory)
stat64("/lib/bash/4.1", 0xbfd3a350)     = -1 ENOENT (No such file or
directory)
open("/lib/libncurses.so.5", O_RDONLY)  = 3

is a part of what happens if I start the MySQL client,

Even starting the bash results in

open("/lib/bash/4.1/tls/i686/sse2/libreadline.so.6", O_RDONLY) = -1
ENOENT (No such file or directory)
stat64("/lib/bash/4.1/tls/i686/sse2", 0xbfe0c4d0) = -1 ENOENT (No such
file or directory)
open("/lib/bash/4.1/tls/i686/libreadline.so.6", O_RDONLY) = -1 ENOENT
(No such file or directory)
stat64("/lib/bash/4.1/tls/i686", 0xbfe0c4d0) = -1 ENOENT (No such file
or directory)
open("/lib/bash/4.1/tls/sse2/libreadline.so.6", O_RDONLY) = -1 ENOENT
(No such file or directory)
stat64("/lib/bash/4.1/tls/sse2", 0xbfe0c4d0) = -1 ENOENT (No such file
or directory)
open("/lib/bash/4.1/tls/libreadline.so.6", O_RDONLY) = -1 ENOENT (No
such file or directory)
stat64("/lib/bash/4.1/tls", 0xbfe0c4d0) = -1 ENOENT (No such file or
directory)
open("/lib/bash/4.1/i686/sse2/libreadline.so.6", O_RDONLY) = -1 ENOENT
(No such file or directory)
stat64("/lib/bash/4.1/i686/sse2", 0xbfe0c4d0) = -1 ENOENT (No such file
or directory)
open("/lib/bash/4.1/i686/libreadline.so.6", O_RDONLY) = -1 ENOENT (No
such file or directory)
stat64("/lib/bash/4.1/i686", 0xbfe0c4d0) = -1 ENOENT (No such file or
directory)
open("/lib/bash/4.1/sse2/libreadline.so.6", O_RDONLY) = -1 ENOENT (No
such file or directory)
stat64("/lib/bash/4.1/sse2", 0xbfe0c4d0) = -1 ENOENT (No such file or
directory)
open("/lib/bash/4.1/libreadline.so.6", O_RDONLY) = -1 ENOENT (No such
file or directory)
stat64("/lib/bash/4.1", 0xbfe0c4d0)     = -1 ENOENT (No such file or
directory)
open("/etc/ld.so.cache", O_RDONLY)      = 3

So can it really be such a huge problem?

Thomas

Two comments. First, your trace isn't showing attempts to open .py
files, it's showing attempts to open the Curses library in the bash
directory. Maybe you also have a problem with the .py files, but it
isn't documented in this trace. It's also not showing the program
that's causing the failing open.

Second, the audit program is an idiot. There are lots of programs
which use the "easier to ask forgiveness" pattern and test for the
existence of optional files by trying to open them. This may be what
Bash is doing.

John Roth

 

[Thomas Rachel]

Am 27.07.2011 14:18 schrieb John Roth:

Two comments. First, your trace isn't showing attempts to open .py
files, it's showing attempts to open the Curses library in the bash
directory.

Of course. My goal was to show that the OP's "problem" is not a python
one, but occurs all over several programs.

Maybe you also have a problem with the .py files,

Why should I?

It's also not showing the program that's causing the failing open.

It is irrelevant, IMO.

It just shows that it seems to be common, even for the bin loader, to
search for libraries to be linked to by trying to open them from several
places. So every program call must be full of "failures" shown by the
"audit program".

Second, the audit program is an idiot. There are lots of programs
which use the "easier to ask forgiveness" pattern and test for the
existence of optional files by trying to open them. This may be what
Bash is doing.

ACK. That is exactly what I wanted to show. (With the difference that
this is probably nt the bash, but the linux loader trying to link a .so,
but for the problem, it doesn't make any difference.)


Thomas

 

[John Roth]

Am 27.07.2011 14:18 schrieb John Roth:


Of course. My goal was to show that the OP's "problem" is not a python
one, but occurs all over several programs.

 > Maybe you also have a problem with the .py files,

Why should I?

 > It's also not showing the program that's causing the failing open.

It is irrelevant, IMO.

It just shows that it seems to be common, even for the bin loader, to
search for libraries to be linked to by trying to open them from several
places. So every program call must be full of "failures" shown by the
"audit program".


ACK. That is exactly what I wanted to show. (With the difference that
this is probably nt the bash, but the linux loader trying to link a .so,
but for the problem, it doesn't make any difference.)

Thomas

Sorry. I thought what you posted was from the OP. I guess I don't
really expect someone to post a completely irrelevant trace in a
thread started by someone who has a problem.

John Roth

 

[Ethan Furman]

Sorry. I thought what you posted was from the OP. I guess I don't
really expect someone to post a completely irrelevant trace in a
thread started by someone who has a problem.

The trace was showing the same results in an effort to demonstrate that
the "problem" was just normal operations. This is very relevant.

~Ethan~

 

[Thomas Rachel]

Am 29.07.2011 14:51 schrieb John Roth:

Sorry. I thought what you posted was from the OP. I guess I don't
really expect someone to post a completely irrelevant trace in a
thread started by someone who has a problem.

In what way do you find the trace irrelevant? It clearly shows that it
is not a good idea for an "audit tool" to report all failing open()
calls and that the OP's "problem" is none, at least not one related to
Python.

No more, no less.


Thomas

 

[Jerry Hill]

Sorry. I thought what you posted was from the OP. I guess I don't
really expect someone to post a completely irrelevant trace in a
thread started by someone who has a problem.

I'm not sure why you would think that that post was by the original
poster, or that it was irrelevant. It seems to me that it
demonstrated exactly the behavior Eldon was complaining about as a
normal part of the operation of lots of normal unix programs. Thomas
even said that in the very first line of his post (after the quoted
bit).

 

[Alan Meyer]

Is there a way to have the Python processor look only for bytecode
files, not .py files? We are seeing huge numbers of Linux audit messages
on production system on which only bytecode files are stored. The audit
subsystem is recording each open failure.

Thanks,
Eldon Ziegler

Here are two suggestions:

1. Put a dummy python file in the directory that does very little,
perhaps something like:

print("%s: You should not be seeing this" % sys.argv[0])

Give it the name of one of the .pyc, e.g., if foo.pyc, then foo.py.

Give the bytecode file a later date, e.g., "touch foo.pyc" (assuming you
can do that without messing up other aspects of your system.)

If that works, then copy the dummy file to each of the other required
..py names in the directory. You'll then have to touch *.pyc to ensure
that the pyc's have later dates. After that, you won't have to do
anything when you replace a .pyc file. When you add a new .pyc you'll
need to do the copy and touch again.

2. Use a log truncator to truncate the audit log.

I don't remember the name of one at the moment, but I believe there is
at least one open source program that will monitor named files and
truncate them from the beginning to a fixed maximum size.

I don't like this method as much as the first because it might result in
something important being truncated.

Alan