Parsing ISO date/time strings - where did the parser go?

Discussion in 'Python' started by John Nagle, Sep 6, 2012.

  1. John Nagle

    John Nagle Guest

    In Python 2.7:

    I want to parse standard ISO date/time strings such as

    2012-09-09T18:00:00-07:00

    into Python "datetime" objects. The "datetime" object offers
    an output method , datetimeobj.isoformat(), but not an input
    parser. There ought to be

    classmethod datetime.fromisoformat(s)

    but there isn't. I'd like to avoid adding a dependency on
    a third party module like "dateutil".

    The "Working with time" section of the Python wiki is so
    ancient it predates "datetime", and says so.

    There's an iso8601 module on PyPi, but it's abandoned; it hasn't been
    updated since 2007 and has many outstanding issues.

    There are mentions of "xml.utils.iso8601.parse" in
    various places, but the "xml" module that comes
    with Python 2.7 doesn't have xml.utils.

    http://www.seehuhn.de/pages/pdate
    says:

    "Unfortunately there is no easy way to parse full ISO 8601 dates using
    the Python standard library."

    It looks like this was taken out of "xml" at some point,
    but not moved into "datetime".

    John Nagle
    John Nagle, Sep 6, 2012
    #1
    1. Advertising

  2. On 09/06/2012 09:27 PM, John Nagle wrote:
    > In Python 2.7:
    >
    > I want to parse standard ISO date/time strings such as
    >
    > 2012-09-09T18:00:00-07:00
    >
    > into Python "datetime" objects. The "datetime" object offers
    > an output method , datetimeobj.isoformat(), but not an input
    > parser. There ought to be
    >
    > classmethod datetime.fromisoformat(s)


    http://docs.python.org/library/datetime.html#datetime.datetime.strptime

    The ISO date/time format is dead simple and well-defined. strptime is
    quite suitable.

    >
    > but there isn't. I'd like to avoid adding a dependency on
    > a third party module like "dateutil".
    >
    > The "Working with time" section of the Python wiki is so
    > ancient it predates "datetime", and says so.
    >
    > There's an iso8601 module on PyPi, but it's abandoned; it hasn't been
    > updated since 2007 and has many outstanding issues.
    >
    > There are mentions of "xml.utils.iso8601.parse" in
    > various places, but the "xml" module that comes
    > with Python 2.7 doesn't have xml.utils.
    >
    > http://www.seehuhn.de/pages/pdate
    > says:
    >
    > "Unfortunately there is no easy way to parse full ISO 8601 dates using
    > the Python standard library."
    >
    > It looks like this was taken out of "xml" at some point,
    > but not moved into "datetime".
    >
    > John Nagle
    >
    Thomas Jollans, Sep 6, 2012
    #2
    1. Advertising

  3. John Nagle

    Paul Rubin Guest

    John Nagle <> writes:
    > There's an iso8601 module on PyPi, but it's abandoned; it hasn't been
    > updated since 2007 and has many outstanding issues.


    Hmm, I have some code that uses ISO date/time strings and just checked
    to see how I did it, and it looks like it uses iso8601-0.1.4-py2.6.egg .
    I don't remember downloading that module (I must have done it and
    forgotten). I'm not sure what its outstanding issues are, as it works
    ok in the limited way I use it.

    I agree that this functionality ought to be in the stdlib.
    Paul Rubin, Sep 6, 2012
    #3
  4. John Nagle

    Dave Angel Guest

    On 09/06/2012 03:27 PM, John Nagle wrote:
    > In Python 2.7:
    >
    > I want to parse standard ISO date/time strings such as
    >
    > 2012-09-09T18:00:00-07:00
    >
    > into Python "datetime" objects. The "datetime" object offers
    > an output method , datetimeobj.isoformat(), but not an input
    > parser. There ought to be
    >
    > classmethod datetime.fromisoformat(s)
    >
    > but there isn't. I'd like to avoid adding a dependency on
    > a third party module like "dateutil".
    >
    > The "Working with time" section of the Python wiki is so
    > ancient it predates "datetime", and says so.
    >
    > There's an iso8601 module on PyPi, but it's abandoned; it hasn't been
    > updated since 2007 and has many outstanding issues.
    >
    > There are mentions of "xml.utils.iso8601.parse" in
    > various places, but the "xml" module that comes
    > with Python 2.7 doesn't have xml.utils.
    >
    > http://www.seehuhn.de/pages/pdate
    > says:
    >
    > "Unfortunately there is no easy way to parse full ISO 8601 dates using
    > the Python standard library."
    >
    > It looks like this was taken out of "xml" at some point,
    > but not moved into "datetime".
    >


    For working with datetime, see
    http://docs.python.org/library/datetime.html#datetime.datetime

    and look up datetime.strptime()

    Likewise for generalized output, check out datetime.strftime().





    --

    DaveA
    Dave Angel, Sep 6, 2012
    #4
  5. John Nagle

    John Nagle Guest

    On 9/6/2012 12:51 PM, Paul Rubin wrote:
    > John Nagle <> writes:
    >> There's an iso8601 module on PyPi, but it's abandoned; it hasn't been
    >> updated since 2007 and has many outstanding issues.

    >
    > Hmm, I have some code that uses ISO date/time strings and just checked
    > to see how I did it, and it looks like it uses iso8601-0.1.4-py2.6.egg .
    > I don't remember downloading that module (I must have done it and
    > forgotten). I'm not sure what its outstanding issues are, as it works
    > ok in the limited way I use it.
    >
    > I agree that this functionality ought to be in the stdlib.


    Yes, it should. There's no shortage of implementations.
    PyPi has four. Each has some defect.

    PyPi offers:

    iso8601 0.1.4 Simple module to parse ISO 8601 dates
    iso8601.py 0.1dev Parse utilities for iso8601 encoding.
    iso8601plus 0.1.6 Simple module to parse ISO 8601 dates
    zc.iso8601 0.2.0 ISO 8601 utility functions

    Unlike CPAN, PyPi has no quality control.

    Looking at the first one, it's in Google Code.

    http://code.google.com/p/pyiso8601/source/browse/trunk/iso8601/iso8601.py

    The first bug is at line 67. For a timestamp with a "Z"
    at the end, the offset should always be zero, regardless of the default
    timezone. See "http://en.wikipedia.org/wiki/ISO_8601".
    The code uses the default time zone in that case, which is wrong.
    So don't call that code with your local time zone as the default;
    it will return bad times.

    Looking at the second one, it's on github:

    https://github.com/accellion/iso8601.py/blob/master/iso8601.py

    Giant regular expressions! The code to handle the offset
    is present, but it doesn't make the datetime object a
    timezone-aware object. It returns a naive object in UTC.

    The third one is at

    https://github.com/jimklo/pyiso8601plus

    This is a fork of the first one, because the first one is abandonware.
    The bug in the first one, mentioned above, isn't fixed. However, if
    a time zone is present, it does return an "aware" datetime object.

    The fourth one is the Zope version. This brings in the pytz
    module, which brings in the Olsen database of named time zones and
    their historical conversion data. None of that information is
    used, or necessary, to parse ISO dates and times. Somebody
    just wanted the pytz.fixedOffset() function, which does something
    datetime already does.

    (For all the people who keep saying "use strptime", that doesn't
    handle time zone offsets at all.)

    John Nagle
    John Nagle, Sep 6, 2012
    #5
  6. John Nagle

    Miki Tebeka Guest

    I'd look also into dateutil.parser.parse and feedparser._parse_date

    On Thursday, September 6, 2012 1:34:18 PM UTC-7, John Nagle wrote:
    > On 9/6/2012 12:51 PM, Paul Rubin wrote:
    >
    > > John Nagle <> writes:

    >
    > >> There's an iso8601 module on PyPi, but it's abandoned; it hasn't been

    >
    > >> updated since 2007 and has many outstanding issues.

    >
    > >

    >
    > > Hmm, I have some code that uses ISO date/time strings and just checked

    >
    > > to see how I did it, and it looks like it uses iso8601-0.1.4-py2.6.egg .

    >
    > > I don't remember downloading that module (I must have done it and

    >
    > > forgotten). I'm not sure what its outstanding issues are, as it works

    >
    > > ok in the limited way I use it.

    >
    > >

    >
    > > I agree that this functionality ought to be in the stdlib.

    >
    >
    >
    > Yes, it should. There's no shortage of implementations.
    >
    > PyPi has four. Each has some defect.
    >
    >
    >
    > PyPi offers:
    >
    >
    >
    > iso8601 0.1.4 Simple module to parse ISO 8601 dates
    >
    > iso8601.py 0.1dev Parse utilities for iso8601 encoding.
    >
    > iso8601plus 0.1.6 Simple module to parse ISO 8601 dates
    >
    > zc.iso8601 0.2.0 ISO 8601 utility functions
    >
    >
    >
    > Unlike CPAN, PyPi has no quality control.
    >
    >
    >
    > Looking at the first one, it's in Google Code.
    >
    >
    >
    > http://code.google.com/p/pyiso8601/source/browse/trunk/iso8601/iso8601.py
    >
    >
    >
    > The first bug is at line 67. For a timestamp with a "Z"
    >
    > at the end, the offset should always be zero, regardless of the default
    >
    > timezone. See "http://en.wikipedia.org/wiki/ISO_8601".
    >
    > The code uses the default time zone in that case, which is wrong.
    >
    > So don't call that code with your local time zone as the default;
    >
    > it will return bad times.
    >
    >
    >
    > Looking at the second one, it's on github:
    >
    >
    >
    > https://github.com/accellion/iso8601.py/blob/master/iso8601.py
    >
    >
    >
    > Giant regular expressions! The code to handle the offset
    >
    > is present, but it doesn't make the datetime object a
    >
    > timezone-aware object. It returns a naive object in UTC.
    >
    >
    >
    > The third one is at
    >
    >
    >
    > https://github.com/jimklo/pyiso8601plus
    >
    >
    >
    > This is a fork of the first one, because the first one is abandonware.
    >
    > The bug in the first one, mentioned above, isn't fixed. However, if
    >
    > a time zone is present, it does return an "aware" datetime object.
    >
    >
    >
    > The fourth one is the Zope version. This brings in the pytz
    >
    > module, which brings in the Olsen database of named time zones and
    >
    > their historical conversion data. None of that information is
    >
    > used, or necessary, to parse ISO dates and times. Somebody
    >
    > just wanted the pytz.fixedOffset() function, which does something
    >
    > datetime already does.
    >
    >
    >
    > (For all the people who keep saying "use strptime", that doesn't
    >
    > handle time zone offsets at all.)
    >
    >
    >
    > John Nagle
    Miki Tebeka, Sep 7, 2012
    #6
  7. John Nagle

    Roy Smith Guest

    In article <k2atej$4rq$>, John Nagle <>
    wrote:

    > In Python 2.7:
    >
    > I want to parse standard ISO date/time strings such as
    >
    > 2012-09-09T18:00:00-07:00
    >
    > into Python "datetime" objects. The "datetime" object offers
    > an output method , datetimeobj.isoformat(), but not an input
    > parser. There ought to be
    >
    > classmethod datetime.fromisoformat(s)
    >
    > but there isn't. I'd like to avoid adding a dependency on
    > a third party module like "dateutil".


    I'm curious why? I really think dateutil is the way to go.

    It's really amazing (and unfortunate) that datetime has isoformat(), but
    no way to go in the other direction.
    Roy Smith, Sep 7, 2012
    #7
  8. John Nagle

    Roy Smith Guest

    In article <>,
    Dave Angel <> wrote:

    > For working with datetime, see
    > http://docs.python.org/library/datetime.html#datetime.datetime
    >
    > and look up datetime.strptime()


    strptime has two problems.

    One is that it's a pain to use (you have to look up all those
    inscrutable %-thingies every time).

    The second is that it doesn't always work. To correctly parse an
    ISO-8601 string, you need '%z', which isn't supported on all platforms.

    The third is that I never use methods I can't figure out how to
    pronounce.
    Roy Smith, Sep 7, 2012
    #8
  9. John Nagle

    Terry Reedy Guest

    On 9/6/2012 3:44 PM, Thomas Jollans wrote:
    > On 09/06/2012 09:27 PM, John Nagle wrote:
    >> In Python 2.7:
    >>
    >> I want to parse standard ISO date/time strings such as
    >>
    >> 2012-09-09T18:00:00-07:00
    >>
    >> into Python "datetime" objects. The "datetime" object offers
    >> an output method , datetimeobj.isoformat(), but not an input
    >> parser. There ought to be
    >>
    >> classmethod datetime.fromisoformat(s)

    >
    > http://docs.python.org/library/datetime.html#datetime.datetime.strptime
    >
    > The ISO date/time format is dead simple and well-defined. strptime is
    > quite suitable.


    I do not see any example formats. An example for ISO might be a good one.

    --
    Terry Jan Reedy
    Terry Reedy, Sep 7, 2012
    #9
  10. * Roy Smith wrote:

    > The third is that I never use methods I can't figure out how to
    > pronounce.


    here: strip'time

    nd
    --
    Flhacs wird im Usenet grundsätzlich alsfhc geschrieben. Schreibt man
    lafhsc nicht slfach, so ist das schlichtweg hclafs. Hingegen darf man
    rihctig ruhig rhitcgi schreiben, weil eine shcalfe Schreibweise bei
    irhictg nicht als shflac angesehen wird. -- Hajo Pflüger in dnq
    André Malo, Sep 8, 2012
    #10
  11. John Nagle

    John Nagle Guest

    On 9/8/2012 5:20 PM, John Gleeson wrote:
    >
    > On 2012-09-06, at 2:34 PM, John Nagle wrote:
    >> Yes, it should. There's no shortage of implementations.
    >> PyPi has four. Each has some defect.
    >>
    >> PyPi offers:
    >>
    >> iso8601 0.1.4 Simple module to parse ISO 8601 dates
    >> iso8601.py 0.1dev Parse utilities for iso8601 encoding.
    >> iso8601plus 0.1.6 Simple module to parse ISO 8601 dates
    >> zc.iso8601 0.2.0 ISO 8601 utility functions

    >
    >
    > Here are three more on PyPI you can try:
    >
    > iso-8601 0.2.3 Flexible ISO 8601 parser...
    > PySO8601 0.1.7 PySO8601 aims to parse any ISO 8601 date...
    > isodate 0.4.8 An ISO 8601 date/time/duration parser and formater
    >
    > All three have been updated this year.


    There's another one inside feedparser, and there used to be
    one in the xml module.

    Filed issue 15873: "datetime" cannot parse ISO 8601 dates and times
    http://bugs.python.org/issue15873

    This really should be handled in the standard library, instead of
    everybody rolling their own, badly. Especially since in Python 3.x,
    there's finally a useful "tzinfo" subclass for fixed time zone
    offsets. That provides a way to directly represent ISO 8601 date/time
    strings with offsets as "time zone aware" date time objects.

    John Nagle
    John Nagle, Sep 9, 2012
    #11
  12. John Nagle

    Roy Smith Guest

    In article <k2h3n2$213$>, John Nagle <>
    wrote:

    > This really should be handled in the standard library, instead of
    > everybody rolling their own, badly.


    +1
    Roy Smith, Sep 9, 2012
    #12
  13. On 09/09/2012 11:15, Roy Smith wrote:
    > In article <k2h3n2$213$>, John Nagle <>
    > wrote:
    >
    >> This really should be handled in the standard library, instead of
    >> everybody rolling their own, badly.

    >
    > +1
    >


    I'll second that given "There should be one-- and preferably only one
    --obvious way to do it".

    --
    Cheers.

    Mark Lawrence.
    Mark Lawrence, Sep 9, 2012
    #13
  14. John Nagle

    Roy Smith Guest

    In article <>,
    Thomas Jollans <> wrote:

    > The ISO date/time format is dead simple and well-defined.


    Well defined, perhaps. But nobody who has read the standard could call
    it "dead simple". ISO-8601-2004(E) is 40 pages long.

    Of course, that fact that it's complicated enough to generate 40 pages
    worth of standards document just argues that much more strongly for it
    being in the standard lib (so there can be one canonical, well-tested,
    way to do it).
    Roy Smith, Sep 9, 2012
    #14
  15. John Nagle

    Pete Forman Guest

    John Nagle <> writes:

    > I want to parse standard ISO date/time strings such as
    >
    > 2012-09-09T18:00:00-07:00
    >
    > into Python "datetime" objects.


    Consider whether RFC 3339 might be a more suitable format.

    It is a subset of ISO 8601 extended format. Some of the restrictions are

    Year must be 4 digits
    Fraction separator is period, not comma
    All components including time-offset are mandatory, except for time-secfrac
    time-minute in time-offset is not optional, must use ±hh:mm or Z

    Some latitude is allowed

    T may be replaced by e.g. space

    Extra feature

    time-offset of -00:00 means UTC but local time is unknown

    --
    Pete Forman
    Pete Forman, Sep 12, 2012
    #15
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Peter Grison

    Date, date date date....

    Peter Grison, May 28, 2004, in forum: Java
    Replies:
    10
    Views:
    3,231
    Michael Borgwardt
    May 30, 2004
  2. Franck DARRAS
    Replies:
    12
    Views:
    623
    Jim Higson
    Aug 23, 2004
  3. Alexei Polkhanov
    Replies:
    11
    Views:
    2,447
  4. Replies:
    13
    Views:
    6,377
    Dave Thompson
    Dec 20, 2004
  5. codejockey
    Replies:
    20
    Views:
    2,874
    James Kanze
    Oct 11, 2010
Loading...

Share This Page