need help with timezone conversion (unexpected side effect oftime.mktime ??)

[Ivan Velev]

Hello,

Minimal example below - it gives me different output if I comment /
uncomment the extra time.mktime call - note that this call is not
related in any way to main logic flow.


When "problematicStamp = ..." is commented I get
gmtStamp: 1130634600.0

when I uncomment that line I get
gmtStamp: 1130631000.0

I have tried this on a couple of Linux machines and it was
reproducible everyewhere. One of those machines has the following
Python version (If needed I can provide more details)

Python 2.5 (r25:51908, Mar 26 2007, 23:34:03)

Any idea what' happening there ?
Ivan



---------------

import time, os

# to see the difference, uncomment this line
# problematicStamp = time.mktime((2004, 10, 30, 4, 10, 0, 6, 303, -1))

os.putenv("TZ", "Europe/Sofia")
time.tzset()

gmtStamp = time.mktime((2005, 10, 30, 3, 10, 0, 6, 303, -1))
print "gmtStamp:", gmtStamp

 

[Paul Boddie]

Minimal example below - it gives me different output if I comment /
uncomment the extra time.mktime call - note that this call is not
related in any way to main logic flow.

When "problematicStamp = ..." is commented I get
gmtStamp: 1130634600.0

when I uncomment that line I get
gmtStamp: 1130631000.0

I've tried this with Python 2.3 and 2.4 on Red Hat Enterprise Linux 4
and can't reproduce the problem, even with other TZ values such as
"EEST3" (which appears to be a legal name and does change the
timestamp produced). I don't think much has changed in the time module
since 2.4, which means that there might be some kind of library or
configuration issue involved which causes the observed behaviour.

Paul

 

[Ivan Velev]

I've tried this with Python 2.3 and 2.4 on Red Hat Enterprise Linux 4

and can't reproduce the problem, even with other TZ values such as

Thanks for the quick reply.

Can you please let me know what value do you receive during your
tests ?


As far as I can see, Python timezone API is just a wrapper around
(shared ) native C libraries, but I really do not have any idea on how
to reproduce this issue in a non-Python environment

btw. problem exists for this particular datetime only, an hour or more
before / after that time givse identical results (this particular
datetime is near the "daylight saving time change" moment)

 

[Paul Boddie]

Thanks for the quick reply.

Can you please let me know what value do you receive during your
tests ?

With "Europe/Sofia" as the time zone, I get 1130634600.0 as the
result. I can't seem to find out what valid values of TZ should be
from the system documentation, but I did find this:

http://www.ludd.luth.se/~ams/djgpp/cvs/djgpp/zoneinfo/src/theory

As far as I can see, Python timezone API is just a wrapper around
(shared ) native C libraries, but I really do not have any idea on how
to reproduce this issue in a non-Python environment

A short C program behaves like Python in this regard:

#include <stdio.h>
#include <time.h>
#include <stdlib.h>

int main(int argc, char *argv[])
{
char tzenv[] = "TZ=Europe/Sofia";
struct tm time_str;
time_t gmtStamp, problematicStamp;

time_str.tm_year = 2005 - 1900;
time_str.tm_mon = 10 - 1;
time_str.tm_mday = 30;
time_str.tm_hour = 3;
time_str.tm_min = 10;
time_str.tm_sec = 0;
time_str.tm_wday = 6;
time_str.tm_yday = 303;
time_str.tm_isdst = -1;
problematicStamp = mktime(&time_str);

putenv(tzenv);
tzset();
time_str.tm_year = 2005 - 1900;
time_str.tm_mon = 10 - 1;
time_str.tm_mday = 30;
time_str.tm_hour = 3;
time_str.tm_min = 10;
time_str.tm_sec = 0;
time_str.tm_wday = 6;
time_str.tm_yday = 303;
time_str.tm_isdst = -1;
gmtStamp = mktime(&time_str);
printf("gmtStamp: %d\n", gmtStamp);
return 0;
}

btw. problem exists for this particular datetime only, an hour or more
before / after that time givse identical results (this particular
datetime is near the "daylight saving time change" moment)

The time functions have behaviour which can be very hard to explain
sometimes. I worked for a while on some patches in order to provide
more reasonable functions for common tasks, but it can be really
difficult to make reliable functions based on the POSIX API, and I'd
recommend the datetime module for any serious work with dates and
times.

Paul

 

[Ivan Velev]

Thanks Paul,

I have identified the "problem" - because of daylight change this
particular timesamp was observed twice in Europe/Sofia. Here is the
GMT-to-local-time conversion:


+------------+---------------------+---------------------+
| gmt_stamp | gmt_time | local_time |
+------------+---------------------+---------------------+
| 1130631000 | 2005-10-30 00:10:00 | 2005-10-30 03:10:00 |
+------------+---------------------+---------------------+
| 1130634600 | 2005-10-30 01:10:00 | 2005-10-30 03:10:00 |
+------------+---------------------+---------------------+
| 1130638200 | 2005-10-30 02:10:00 | 2005-10-30 04:10:00 |
+------------+---------------------+---------------------+
| 1130641800 | 2005-10-30 03:10:00 | 2005-10-30 05:10:00 |
+------------+---------------------+---------------------+

When you do local-time-to-GMT conversion you can expect any of those
two timestamps. I missed that initially (I was sure that local time
has "one hour gap" and not "one hour of overlapping time")

and I'd recommend the datetime module for any serious work with dates and times.

Last time when I was playing with TZ conversions, I was not able to do
anything using datetime module - it seems that one needs to define his
own set of timezones (+ all the details) to get it working ... Am I
wrong ? Can you show me how do accomplish the same conversion using
datetime module ?

Thanks again,
Ivan

 

[Paul Boddie]

Thanks Paul,

I have identified the "problem" - because of daylight change this
particular timesamp was observed twice in Europe/Sofia. Here is the
GMT-to-local-time conversion:

+------------+---------------------+---------------------+
| gmt_stamp | gmt_time | local_time |
+------------+---------------------+---------------------+
| 1130631000 | 2005-10-30 00:10:00 | 2005-10-30 03:10:00 |
+------------+---------------------+---------------------+
| 1130634600 | 2005-10-30 01:10:00 | 2005-10-30 03:10:00 |
+------------+---------------------+---------------------+
| 1130638200 | 2005-10-30 02:10:00 | 2005-10-30 04:10:00 |
+------------+---------------------+---------------------+
| 1130641800 | 2005-10-30 03:10:00 | 2005-10-30 05:10:00 |
+------------+---------------------+---------------------+

I still don't understand why the "problematic" timestamp would affect
the latter operation, though. I can see that both timestamps could be
valid depending on which time zone is supposed to be in operation -
have the dates for daylight saving (summer vs. winter) time changed in
Bulgaria in the last few years? Maybe asking for a conversion for a
date in 2004 invokes some old rules which then affect a conversion for
a date in 2005, even though that would be really bad behaviour (which
I don't see on this machine here).

When you do local-time-to-GMT conversion you can expect any of those
two timestamps. I missed that initially (I was sure that local time
has "one hour gap" and not "one hour of overlapping time")

Well, it really isn't overlapping as such: you're in different zones,
of course. ;-)

Last time when I was playing with TZ conversions, I was not able to do
anything using datetime module - it seems that one needs to define his
own set of timezones (+ all the details) to get it working ... Am I
wrong ? Can you show me how do accomplish the same conversion using
datetime module ?

I think it's easiest to install one of the libraries which provides
the zone data. The first one that comes to mind is python-dateutil,
but I think there are others.

Paul

 

[Ivan Velev]

I still don't understand why the "problematic" timestamp would affect

the latter operation, though. I can see that both timestamps could be
valid depending on which time zone is supposed to be in operation -
have the dates for daylight saving (summer vs. winter) time changed in
Bulgaria in the last few years? Maybe asking for a conversion for a
date in 2004 invokes some old rules which then affect a conversion for
a date in 2005, even though that would be really bad behaviour (which
I don't see on this machine here).

According to this page:

http://www.timeanddate.com/worldclock/timezone.html?n=238

In 2004 daylight change was done on October 31, while in 2005 it was
done on October 30. Maybe this fact cause the difference in "guess
what is the current daylight offset" behavior ??

Well, it really isn't overlapping as such: you're in different zones,
of course. ;-)

Heh, got it ... apparently "Europe/Sofia" is not a single timezone, it
is an alias for two timezones

I think it's easiest to install one of the libraries which provides
the zone data. The first one that comes to mind is python-dateutil,
but I think there are others.

Thanks I will try it.

Once again, thanks for your helpfull responses !
Ivan