os.stat bug?

Discussion in 'Python' started by Laszlo Nagy, Mar 21, 2011.

  1. Laszlo Nagy

    Laszlo Nagy Guest

    Hi All,

    I have a Python program that goes up to 100% CPU. Just like this (top):

    PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU
    COMMAND
    80212 user1 2 44 0 70520K 16212K select 1 0:30 100.00%
    /usr/local/bin/python process_updates_ss_od.py -l 10

    I have added extra logs and it turns out that there are two threads. One
    thread is calling "time.sleep()" and the other is calling "os.stat"
    call. (Actually it is calling os.path.isfile, but I hunted down the last
    link in the chain.) The most interesting thing is that the process is in
    "SELECT" state. As far as I know, CPU load should be 0% because "select"
    state should block program execution until the I/O completes.

    I must also tell you that the os.stat call is taking long because this
    system has about 7 million files on a slow disk. It would be normal for
    an os.stat call to return after 10 seconds. I have no problem with that.
    But I think that the 100% CPU is not acceptable. I guess that the code
    is running in kernel mode. I think this because I can send a KILL signal
    to it and the state changes to the following:


    PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU
    COMMAND
    80212 user1 2 44 0 70520K 15256K STOP 5 1:27 100.00%
    /usr/local/bin/python process_updates_ss_od.py -l 10

    So the state of the process changes to "STOP", but the program does not
    stop until the os.stat call returns back (sometimes for 30 seconds).

    Could it be a problem with the operation system? Is it possible that an
    os.stat call requires 100% CPU power from the OS? Or is it a problem
    with the Python implementation?

    (Unfortunately I cannot give you an example program. Giving an example
    would require giving you a slow I/O device with millions of files on it.)

    OS version: FreeBSD 8.1-STABLE amd64
    Python version: 2.6.6

    Thanks,

    Laszlo


    --
    This message has been scanned for viruses and
    dangerous content by MailScanner, and is
    believed to be clean.
     
    Laszlo Nagy, Mar 21, 2011
    #1
    1. Advertising

  2. > Could it be a problem with the operation system? Is it possible that an
    > os.stat call requires 100% CPU power from the OS? Or is it a problem
    > with the Python implementation?


    There is a chance that the CPU usage actually comes from the thread
    doing sleep(). If you have a very short sleep time, and a loop around
    it, it may spin in this loop.

    If it's not that, and if it's not any other unrelated application that
    uses CPU that you didn't mention, then chances are high that it's indeed
    the file system code of your operating system that consumes that much
    CPU time.

    Regards,
    Martin
     
    Martin v. Loewis, Mar 21, 2011
    #2
    1. Advertising

  3. Laszlo Nagy

    Laszlo Nagy Guest

    On 2011-03-21 10:58, Martin v. Loewis wrote:
    >> Could it be a problem with the operation system? Is it possible that an
    >> os.stat call requires 100% CPU power from the OS? Or is it a problem
    >> with the Python implementation?

    > There is a chance that the CPU usage actually comes from the thread
    > doing sleep(). If you have a very short sleep time, and a loop around
    > it, it may spin in this loop.

    Sleep time is 1.0 sec.
    > If it's not that, and if it's not any other unrelated application that
    > uses CPU that you didn't mention, then chances are high that it's indeed
    > the file system code of your operating system that consumes that much
    > CPU time.

    Yes. top shows 2 thread in this process, and full 100% CPU time is
    dedicated to the process. So it must be it.

    Thank you,

    Laszlo
     
    Laszlo Nagy, Mar 21, 2011
    #3
  4. Laszlo Nagy

    Nobody Guest

    On Mon, 21 Mar 2011 09:32:11 +0100, Laszlo Nagy wrote:

    > So the state of the process changes to "STOP", but the program does not
    > stop until the os.stat call returns back (sometimes for 30 seconds).
    >
    > Could it be a problem with the operation system? Is it possible that an
    > os.stat call requires 100% CPU power from the OS? Or is it a problem
    > with the Python implementation?


    It's the OS kernel. If it was Python or the C library, sending SIGKILL
    would result in immediate termination.

    Is the disk interface operating in PIO mode? A slow disk shouldn't cause
    100% CPU consumption; the OS would just get on with something else (or
    just idle) while waiting for data to become available. But if it's
    having to copy data from the controller one word at a time, that could
    cause it (and would also make the disk appear slow).
     
    Nobody, Mar 21, 2011
    #4
  5. Laszlo Nagy

    Laszlo Nagy Guest


    > It's the OS kernel. If it was Python or the C library, sending SIGKILL
    > would result in immediate termination.
    >
    > Is the disk interface operating in PIO mode? A slow disk shouldn't cause
    > 100% CPU consumption; the OS would just get on with something else (or
    > just idle) while waiting for data to become available. But if it's
    > having to copy data from the controller one word at a time, that could
    > cause it (and would also make the disk appear slow).
    >

    This is a RAID 1+0 array with 10 hard disks plus a SCSI controller with
    2GB write back cache. I don't think that it has to do anything with disk
    speed. The CPU load goes up to 100% (the disk I/O does not go up that much).

    I'm working about a different storage method. We will be storing these
    logs in a real database instead of separate CSV files. So probably this
    problem will cease.

    Thanks,

    Laszlo
     
    Laszlo Nagy, Mar 24, 2011
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Steven T. Hatton
    Replies:
    3
    Views:
    9,712
    Pete Becker
    Jun 2, 2005
  2. Patrick Useldinger

    os.stat('<filename>')[stat.ST_INO] on Windows

    Patrick Useldinger, Feb 27, 2005, in forum: Python
    Replies:
    6
    Views:
    1,176
    =?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=
    Mar 3, 2005
  3. Magesh
    Replies:
    3
    Views:
    444
    Gordon Burditt
    Oct 5, 2007
  4. Rolf Krüger
    Replies:
    2
    Views:
    616
    Ian Collins
    Mar 12, 2008
  5. ruck
    Replies:
    10
    Views:
    1,278
Loading...

Share This Page