Determining actual elapsed (wall-clock) time

Discussion in 'Python' started by Peter Hansen, Jul 2, 2005.

  1. Peter Hansen

    Peter Hansen Guest

    I would like to determine the "actual" elapsed time of an operation
    which could take place during a time change, in a platform-independent
    manner (at least across Linux/Windows machines).

    Using time.time() doesn't appear to be suitable, since time might jump
    forwards or backwards at the user's whim, if the system clock is reset,
    or when a daylight savings time change occurs.

    Using time.clock() doesn't appear to be suitable on Linux, though it
    appears sufficient on Windows. (I'm willing to assume nobody will reset
    whatever QueryPerformanceCounter() uses to return the time.clock() value
    on Windows.)

    There used to be a "timing" module (according to
    http://effbot.org/librarybook/timing.htm) which would likely have worked
    (for Unix at least), but it's not in the latest Pythons.

    I can deal with wraparounds if necessary. I'm not too concerned about
    the resolution, though better than one second would be useful.

    Thanks for any suggestions.

    -Peter
     
    Peter Hansen, Jul 2, 2005
    #1
    1. Advertising

  2. Peter Hansen

    Roy Smith Guest

    Peter Hansen <> wrote:
    > I would like to determine the "actual" elapsed time of an operation
    > which could take place during a time change, in a platform-independent
    > manner (at least across Linux/Windows machines).
    >
    > Using time.time() doesn't appear to be suitable, since time might jump
    > forwards or backwards at the user's whim, if the system clock is reset,
    > or when a daylight savings time change occurs.


    If you get the UTC time, daylight savings time doesn't enter the equation.
    If the system clock is reset, however, you're out of luck. I can't think
    of any time-related API which doesn't rely on the system clock as a
    reference. If the system clock is good, you get good time. If the system
    clock sucks, or changes, you don't.

    If you care about time, you want your system clock controlled by NTP.
    There's just no excuse not to.

    Is there some reason you can't just use the system clock? I suppose if you
    had to, you could hobble together your own NTP client which keeps network
    time independent of the system clock. But that would be a lot of work and
    it's hard to imagine the effort would be justified.
     
    Roy Smith, Jul 2, 2005
    #2
    1. Advertising

  3. Peter Hansen

    Peter Hansen Guest

    Roy Smith wrote:
    > Peter Hansen <> wrote:
    > If you get the UTC time, daylight savings time doesn't enter the equation.


    Of course... I didn't think of that approach. I don't actually care
    about absolute time, so this should work fine for at least the DST case.

    > If you care about time, you want your system clock controlled by NTP.
    > There's just no excuse not to.


    I guess as long as the NTP client is set up to ensure the time
    adjustments are smaller than some value X, it would be acceptable. I'll
    have to look into how to set up Windows XP to prevent users from
    changing the time on their own, assuming that's possible.

    > Is there some reason you can't just use the system clock?


    I was anticipating use of the system (running on Windows in this case)
    by users who might decide to change the time, and in certain cases that
    could cripple the software. I hadn't thought of the possibility of
    simply preventing that, though if I insist on an NTP client being
    installed, they shouldn't have a problem with that.

    Thanks for correcting my thought process...

    -Peter
     
    Peter Hansen, Jul 2, 2005
    #3
  4. Peter Hansen

    Roy Smith Guest

    Peter Hansen <> wrote:
    > I guess as long as the NTP client is set up to ensure the time
    > adjustments are smaller than some value X, it would be acceptable.


    NTP is generally capable of keeping the various system clocks on a LAN
    within a few ms of each other, and within a few 10's of ms over the
    Internet from GPS, WWV, or similar international time references.

    > I'll have to look into how to set up Windows XP to prevent users from
    > changing the time on their own, assuming that's possible.


    On a single-user system like Windows, you pretty much have to assume the
    user can do anything. They can turn off NTP, reset the clock, reboot the
    system, uninstall your software, whatever.

    If you could check to see that NTP is running, it doesn't prove anything.
    A malicious and determined user could set up another machine as a NTP
    server to synch against, and even configure that machine to look like it
    was a stratum-1 reference (i.e. an atomic clock).

    At some point, you need to decide if you trust the system administrator to
    supply you with an accurate system clock or not. If you don't, and it's
    really important that you have an accurate time reference, you've got an
    interesting engineering problem on your hands.
     
    Roy Smith, Jul 2, 2005
    #4
  5. Peter Hansen

    Peter Hansen Guest

    Roy Smith wrote:
    > Peter Hansen <> wrote:
    >>I'll have to look into how to set up Windows XP to prevent users from
    >>changing the time on their own, assuming that's possible.

    >
    > On a single-user system like Windows, you pretty much have to assume the
    > user can do anything. They can turn off NTP, reset the clock, reboot the
    > system, uninstall your software, whatever.


    Actually, I suspect that a non-administrator user can be restricted from
    doing several of those things under XP, which would simplify things.

    And, in any case, I'm not remotely concerned about malicious users, just
    "fiddlers" who might think their watch time is somehow more correct than
    what the computer says...

    -Peter
     
    Peter Hansen, Jul 2, 2005
    #5
  6. Peter Hansen

    Peter Hansen Guest

    Roy Smith wrote:
    > Peter Hansen <> wrote:
    >>Using time.time() doesn't appear to be suitable, since time might jump
    >>forwards or backwards at the user's whim, if the system clock is reset,
    >>or when a daylight savings time change occurs.

    >
    > If you get the UTC time, daylight savings time doesn't enter the equation.


    Hmmm... not only that, but at least under XP the return value of
    time.time() _is_ UTC. At least, it's entirely unaffected by the
    daylight savings time change, or (apparently) by changes in time zone.

    Looks like time.time() might be sufficient. (That might be what Roy
    intended, but I initially thought he meant time.gmtime(time.time()),
    which could have resulted in a race condition if time.time() jumped
    ahead or back separately from some internal daylight savings time flag,
    and the value was read between the two operations.)

    -Peter
     
    Peter Hansen, Jul 3, 2005
    #6
  7. Peter Hansen

    Tim Peters Guest

    [Peter Hansen]
    > Hmmm... not only that, but at least under XP the return value of
    > time.time() _is_ UTC. At least, it's entirely unaffected by the
    > daylight savings time change, or (apparently) by changes in time zone.


    On all platforms, time.time() returns the number of seconds "since the
    epoch". All POSIX systems agree on when "the epoch" began, but that
    doesn't really matter to your use case. Number of seconds since the
    epoch is insensitive to daylight time, time zone, leap seconds, etc.
    Users can nevertheless make it appear to jump (into the future or the
    past) by changing their system clock. If you need an absolute measure
    of time immune to user whims, you need to connect to special hardware,
    or to an external time source.
     
    Tim Peters, Jul 3, 2005
    #7
  8. Peter Hansen

    John Machin Guest

    Roy Smith wrote:
    > Peter Hansen <> wrote:
    >
    >>I guess as long as the NTP client is set up to ensure the time
    >>adjustments are smaller than some value X, it would be acceptable.

    >
    >
    > NTP is generally capable of keeping the various system clocks on a LAN
    > within a few ms of each other, and within a few 10's of ms over the
    > Internet from GPS, WWV, or similar international time references.
    >
    >
    >>I'll have to look into how to set up Windows XP to prevent users from
    >>changing the time on their own, assuming that's possible.

    >
    >
    > On a single-user system like Windows, you pretty much have to assume the
    > user can do anything. They can turn off NTP, reset the clock, reboot the
    > system, uninstall your software, whatever.
    >
    > If you could check to see that NTP is running, it doesn't prove anything.
    > A malicious and determined user could set up another machine as a NTP
    > server to synch against, and even configure that machine to look like it
    > was a stratum-1 reference (i.e. an atomic clock).
    >
    > At some point, you need to decide if you trust the system administrator to
    > supply you with an accurate system clock or not. If you don't, and it's
    > really important that you have an accurate time reference, you've got an
    > interesting engineering problem on your hands.


    A couple of quick thoughts: (1) stupidity is much more prevalent than
    malice in that environment.

    (2) Peter, if your app has something else to measure e.g. it is
    processing zillions of rows from a database, grab the [UTC] wall time
    every N things, and apply plausibility checks to the speed N/delta(wall)
    -- if it goes negative or "too high" or "too slow", holler fer a mountie.
     
    John Machin, Jul 3, 2005
    #8
  9. On Sat, 02 Jul 2005 18:55:46 -0400, Peter Hansen <>
    declaimed the following in comp.lang.python:


    > And, in any case, I'm not remotely concerned about malicious users, just
    > "fiddlers" who might think their watch time is somehow more correct than
    > what the computer says...
    >

    In my case, it often is... I've got four WWVB clocks in my
    apartment (one wall powered with LASER/LED projector to the ceiling, one
    part of a "weather station" that also reads temp/humidity from three
    remotes, a battery alarm clock, and a travel clock), along with two GPS
    units, and a DVD-R and VCR that time synch to PBS signals... My watch
    tends to be less than 20 seconds off when the 6-month DST change takes
    place. My computers tend to be a few minutes off in that time span.

    --
    > ============================================================== <
    > | Wulfraed Dennis Lee Bieber KD6MOG <
    > | Bestiaria Support Staff <
    > ============================================================== <
    > Home Page: <http://www.dm.net/~wulfraed/> <
    > Overflow Page: <http://wlfraed.home.netcom.com/> <
     
    Dennis Lee Bieber, Jul 3, 2005
    #9
  10. Peter Hansen

    flamesrock Guest

    I've needed to do something similar in the past and used free ntp
    servers. Of course you need an internet connection:
    http://ntp.isc.org/bin/view/Servers/NTPPoolServers
    http://www.nightsong.com/phr/python/setclock.py

    You could also have a startup script spawn a thread that stores the
    time persistently in a shelf object, sleeps for a minute, and checks to
    see if the current clock time is greater than a minute (+/-sec,) and
    subtracts the difference. Or even check it against an NTP server if you
    can.
     
    flamesrock, Jul 3, 2005
    #10
  11. On Sat, 2 Jul 2005 19:44:19 -0400, Tim Peters <> wrote:

    >[Peter Hansen]
    >> Hmmm... not only that, but at least under XP the return value of
    >> time.time() _is_ UTC. At least, it's entirely unaffected by the
    >> daylight savings time change, or (apparently) by changes in time zone.

    >
    >On all platforms, time.time() returns the number of seconds "since the
    >epoch". All POSIX systems agree on when "the epoch" began, but that
    >doesn't really matter to your use case. Number of seconds since the
    >epoch is insensitive to daylight time, time zone, leap seconds, etc.=20
    >Users can nevertheless make it appear to jump (into the future or the
    >past) by changing their system clock. If you need an absolute measure
    >of time immune to user whims, you need to connect to special hardware,
    >or to an external time source.


    For the latter, Peter, you can probably adapt Paul Rubin' setclock.py
    found at
    http://www.nightsong.com/phr/python/setclock.py

    Regards,
    Bengt Richter
     
    Bengt Richter, Jul 3, 2005
    #11
  12. Peter Hansen

    Aahz Guest

    In article <>,
    Peter Hansen <> wrote:
    >
    >I would like to determine the "actual" elapsed time of an operation
    >which could take place during a time change, in a platform-independent
    >manner (at least across Linux/Windows machines).
    >
    >Using time.time() doesn't appear to be suitable, since time might jump
    >forwards or backwards at the user's whim, if the system clock is reset,
    >or when a daylight savings time change occurs.


    You have to do one of two things: rely on system clock or require a
    network connection (so you can call a timeserver directly).
    --
    Aahz () <*> http://www.pythoncraft.com/

    f u cn rd ths, u cn gt a gd jb n nx prgrmmng.
     
    Aahz, Jul 4, 2005
    #12
  13. Peter Hansen

    zooko Guest

    The traditional use of gettimeofday() to (insecurely and unreliably)
    approximate elapsed local time is one of my pet peeves.

    Fortunately a real monotonic clock has finally been added to the linux
    kernel and glibc:

    http://www.imperialviolet.org/page24.html#e474

    If you have a recent enough kernel and glibc you can use that. On
    Windows you can use system timers and handle wraparound yourself.

    And you can write an abstract Pythonic layer over both and give it to
    me. ;-)
     
    zooko, Jul 7, 2005
    #13
  14. Peter Hansen

    Peter Hansen Guest

    zooko wrote:
    > The traditional use of gettimeofday() to (insecurely and unreliably)
    > approximate elapsed local time is one of my pet peeves.
    >
    > Fortunately a real monotonic clock has finally been added to the linux
    > kernel and glibc:
    >
    > http://www.imperialviolet.org/page24.html#e474


    Interestingly, the author of that page appears to have made a number of
    the same misassumptions about the actual behaviour of the timeofday
    clock (assuming time.time() fairly faithfully reproduces/wraps that
    behaviour). Either that or time.time() does magic that avoids the bulk
    of the problems in which case I can say only "sucks not to be using
    Python, doesn't it?"

    I'd be more interested in this monotonic clock feature if it were not
    the case that time.time() is pretty much monotonic except in the face of
    the minor (sub-100ms) tweaks that might occur once every week or two
    with an NTP client running. It certainly doesn't cause the one hour
    jumps forwards and backwards which I and the author of that page both
    thought it would.

    -Peter
     
    Peter Hansen, Jul 7, 2005
    #14
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Coates, Steve (ACHE)

    RE: Determining actual elapsed (wall-clock) time

    Coates, Steve (ACHE), Jul 4, 2005, in forum: Python
    Replies:
    2
    Views:
    404
    Roy Smith
    Jul 4, 2005
  2. Geert Jansen
    Replies:
    1
    Views:
    337
    Dieter Maurer
    Nov 26, 2005
  3. aegis

    wall clock time

    aegis, Dec 15, 2004, in forum: C Programming
    Replies:
    12
    Views:
    675
    infobahn
    Dec 18, 2004
  4. Replies:
    5
    Views:
    1,630
    Ben Bacarisse
    Nov 19, 2006
  5. rutherf
    Replies:
    2
    Views:
    462
    rutherf
    Oct 28, 2006
Loading...

Share This Page