Parse timestamp string

[Reiner Merz]

Hi,

I'm looking for advice on how to parse a timestamp string
according to the ISO 8601 specification.

For those unfamiliar with the standard, here's an example:

2003-09-09T23:00:00Z

A compact form of it can as well be without the minuses and
the colons or with timezone information.

The parser should be small and fast and fairly easy to
integrate into C. And if possible, I don't want to link to an
additional library.

I thought so far about doing it in lex/yacc (flex/bison) but
after thinking it through, I don't consider them to be the right
tools for the job.

Regexes would be exactly what I need but I don't want to have
a whole regex engine that calculates the pattern at runtime when
it's not necessary. There must be a cheaper way of doing it.

I've already searched the internet for a kind of regex
compiler/generator that would generate C code for my specific
regex but unfortunately I didn't find anything similar to what I
want.

Of course, I can do it the hard way by messing around with
the string functions in string.h but if I can avoid that I'll do
so.

Any suggestions are appreciated.

Thanks

Reiner

 

[Bertrand Mollinier Toublet]

Hi,

I'm looking for advice on how to parse a timestamp string
according to the ISO 8601 specification.

For those unfamiliar with the standard, here's an example:

2003-09-09T23:00:00Z

Of course, I can do it the hard way by messing around with
the string functions in string.h but if I can avoid that I'll do
so.

I do not really think this is such a big deal and given your
requirements, that, er, I snipped, I think this is the way to go.

Now, depending on the applications that you have in mind for this piece
of code, I am more than willing to develop that for you for a reasonable
fee. Feel free to contact me directly if you are interested.

 

[Nick Austin]

Hi,

I'm looking for advice on how to parse a timestamp string
according to the ISO 8601 specification.

For those unfamiliar with the standard, here's an example:

2003-09-09T23:00:00Z

A compact form of it can as well be without the minuses and
the colons or with timezone information.

The parser should be small and fast and fairly easy to
integrate into C. And if possible, I don't want to link to an
additional library.

C already includes libraries that include pattern matching. The
example you give can be parsed with:

sscanf( string, "%d-%d-%dT%d:%d:%d%c", /* ... */ );

You need to look at the return code to determine whether the
match was successful or not.

Start with one of the longer patterns first and try each pattern
until you find the one that succeeds.

Nick.

 

[Al Bowers]

Hi,

I'm looking for advice on how to parse a timestamp string
according to the ISO 8601 specification.

For those unfamiliar with the standard, here's an example:

2003-09-09T23:00:00Z

A compact form of it can as well be without the minuses and
the colons or with timezone information.

The parser should be small and fast and fairly easy to
integrate into C. And if possible, I don't want to link to an
additional library.

I thought so far about doing it in lex/yacc (flex/bison) but
after thinking it through, I don't consider them to be the right
tools for the job.

Regexes would be exactly what I need but I don't want to have
a whole regex engine that calculates the pattern at runtime when
it's not necessary. There must be a cheaper way of doing it.

I've already searched the internet for a kind of regex
compiler/generator that would generate C code for my specific
regex but unfortunately I didn't find anything similar to what I
want.

Of course, I can do it the hard way by messing around with
the string functions in string.h but if I can avoid that I'll do
so.

Any suggestions are appreciated.

You may remove the possible non-digits from the timestamp with
a routine like

char *str = "2003-09-09T23:00:00Z ", buf[32];
int i,j;

for(i = 0,j = 0;str != '\0';i++)
if(str >= '0' && str <= '9')
buf[j++] = str;
buf[j] = '\0';

buf would then be "20030909230000"
Then function sscanf to parse using the format string
"%4u%2u%2u%2u%u%2u"

include <stdio.h>
unsigned yr, mon, day, hr, min, sec

sscanf(buf,"%4u%2u%2u%2u%u%2u",&yr,&mon,&day,&hr,&min,&sec);

 

[Michel Bardiaux]

>
>
> You may remove the possible non-digits from the timestamp with a
> routine like
>
> char *str = "2003-09-09T23:00:00Z ", buf[32]; int i,j;
>
> for(i = 0,j = 0;str != '\0';i++) if(str >= '0' && str <=
> '9') buf[j++] = str; buf[j] = '\0';
>
> buf would then be "20030909230000"

IIRC ISO8601 requires the T and the Z. You can only remove the - and the
:. And they must be all absent or all present, ie "200309-09T23:00:00Z"
is *NOT* acceptable.

> Then function sscanf to parse using the format string
> "%4u%2u%2u%2u%u%2u"
>
> include <stdio.h> unsigned yr, mon, day, hr, min, sec
>
> sscanf(buf,"%4u%2u%2u%2u%u%2u",&yr,&mon,&day,&hr,&min,&sec);
>

The big problem with a scanf implementation is that it does not tell you
where an illegal string fails.

 

[Dave Thompson]

IIRC ISO8601 requires the T and the Z. You can only remove the - and the
:. And they must be all absent or all present, ie "200309-09T23:00:00Z"
is *NOT* acceptable.

In the interchange form yes, but you can do what you like internally.
Obviously the fifth conversion should be %2u in both of those.

The big problem with a scanf implementation is that it does not tell you
where an illegal string fails.

The return value tells you *at which conversion* it failed, and if you
need to know the exact character you can intersperse %n's.

- David.Thompson1 at worldnet.att.net

 

[Reiner Merz]

Thank you all for your contributions.

Unfortunately, this is not what I want but I must admit, I
haven't made myself clear enough.

Your parsing methods are all simple and working for complete
timestamp strings. But sscanf() is just too tolerant in my C
compiler (gcc) to be useful to me.

For example:

If I parse the string

03-09-09T23:00:00Z

"%4d" doesn't return an error on the year because it's not 4
digits long, which I really need.

So, I need a precise scanner function with the parser telling
me exacly where an error occured.

Thanks,

Reiner

 

[Irrwahn Grausewitz]

Thank you all for your contributions.

Unfortunately, this is not what I want but I must admit, I
haven't made myself clear enough.

Your parsing methods are all simple and working for complete
timestamp strings. But sscanf() is just too tolerant in my C
compiler (gcc) to be useful to me.

<SNIP>

Hmm, well, it seems you are in trouble now.
Let me put it together:

You don't want to:
- link against an external library
- use lex/yacc (which, IMHO, are well suited to produce a
scanner/parser; that's what they're made for)
- use any other third party universal regex driven parsing engine
- "mess around" with C's string functions
- use sscanf (for good reasons, IMO)

IMHO you have to drop at least one of these restrictions to accomplish
your mission. Choose the solution that looks least worse to you.

My two-cent suggestion: use (f)lex/yacc(bison) - this way your
scanner/parser will be easier to maintain than any hand crafted
solution.

Good luck to you!

Regards

Irrwahn

 

[Mark McIntyre]

If I parse the string

03-09-09T23:00:00Z

"%4d" doesn't return an error on the year because it's not 4
digits long, which I really need.

So, I need a precise scanner function with the parser telling
me exacly where an error occured.

read the string char by char. Decide what you have as you receive it.

eg
read 2 chars
if next char is not numeric, then add 2000 to first two to form year
etc

 

[Reiner Merz]

Hi Irrwahn,

You don't want to:
- use lex/yacc (which, IMHO, are well suited to produce a
scanner/parser; that's what they're made for)

You must have got me wrong there.

I did try to use lex/yacc because I thought they were made
for these kind of problems, but I didn't succeed. But my
inability to do it could be due to the fact that I've only
rarely used lex/yacc and when I used it, I used it for simple
configuration files. So, you could say, I'm not really
experienced.

My problem was mainly the scanner. I wanted to define my tokens

YEAR as four digits
MONTH as two digits
DAY as two digits
HOUR as two digits
MINUTE as two digits
SECOND as two digits

So, how can the scanner possibly distinguish between MONTH,
DAY, HOUR, MINUTE and SECOND? The flex manual says it picks the
longest possible match and if there are equally long matches it
picks the first defined token.

Ok, I could define a FOURDIGIT and a TWODIGIT token and then
assign the right values in bison. I guess that would work for

2003-09-09T23:00:00Z

but for

20030909T230000Z (which is a legal date time according to the
ISO specification)

I would imagine, lex gives me

FOURDIGIT 2003
FOURDIGIT 0909
FOURDIGIT 2300
TWODIGIT 00

and that is not really what I want.

But maybe I didn't understand the way lex/yacc work.

Cheers,

Reiner

 

[Irrwahn Grausewitz]

Hi Irrwahn,


You must have got me wrong there. Possibly ...

I did try to use lex/yacc because I thought they were made
for these kind of problems, but I didn't succeed. But my
inability to do it could be due to the fact that I've only
rarely used lex/yacc and when I used it, I used it for simple
configuration files. So, you could say, I'm not really
experienced.

My problem was mainly the scanner. I wanted to define my tokens

YEAR as four digits
MONTH as two digits
DAY as two digits
HOUR as two digits
MINUTE as two digits
SECOND as two digits

So, how can the scanner possibly distinguish between MONTH,
DAY, HOUR, MINUTE and SECOND? The flex manual says it picks the
longest possible match and if there are equally long matches it
picks the first defined token.

Ok, I could define a FOURDIGIT and a TWODIGIT token and then
assign the right values in bison. I guess that would work for

2003-09-09T23:00:00Z

but for

20030909T230000Z (which is a legal date time according to the
ISO specification)

Blame me, I was only thinking about the hyphen-separated format.
(And, besides, IMO the latter format is sheer braindead.)

I would imagine, lex gives me

FOURDIGIT 2003
FOURDIGIT 0909
FOURDIGIT 2300
TWODIGIT 00

and that is not really what I want.

Well, obviously not, and furthermore, I cannot imagine a way to tell lex
how to disambiguate this and tokenize appropriately; but I am far from
being a lex expert.

But maybe I didn't understand the way lex/yacc work.

Possibly better than I do.

However, to get back to your original problem, maybe the easiest
solution is to handcraft a scanner, traversing the string and use the
is*() functions from ctype.h, a la:

- read four consecutive digits, interprete as year
- gulp an optional hyphen
- read two consecutive digits, interprete as month
- gulp an optional hyphen
- read two consecutive digits, interprete as day
- gulp the mandatory 'T'
- read two consecutive digits, interprete as hour
- gulp an optional colon
- ... etc.

Not really a nice thing to do I admit, and you have to provide a lot of
error checking; but, after all, you know exactly the way your code
works.

Sorry, I'm unable to provide a better solution, maybe someone else can.
Good luck to you!

Regards

Irrwahn

 

[Kevin Easton]

Hi Irrwahn,


You must have got me wrong there.

I did try to use lex/yacc because I thought they were made
for these kind of problems, but I didn't succeed. But my
inability to do it could be due to the fact that I've only
rarely used lex/yacc and when I used it, I used it for simple
configuration files. So, you could say, I'm not really
experienced.

My problem was mainly the scanner. I wanted to define my tokens

YEAR as four digits
MONTH as two digits
DAY as two digits
HOUR as two digits
MINUTE as two digits
SECOND as two digits

So, how can the scanner possibly distinguish between MONTH,
DAY, HOUR, MINUTE and SECOND? The flex manual says it picks the
longest possible match and if there are equally long matches it
picks the first defined token.

Ok, I could define a FOURDIGIT and a TWODIGIT token and then
assign the right values in bison. I guess that would work for

2003-09-09T23:00:00Z

but for

20030909T230000Z (which is a legal date time according to the
ISO specification)

I would imagine, lex gives me

FOURDIGIT 2003
FOURDIGIT 0909
FOURDIGIT 2300
TWODIGIT 00

and that is not really what I want.

comp.compilers is probably a much better place to ask lex/yacc
questions. However, I can tell you that the solution is to just
define a single DIGIT token, then use yacc to recognise four DIGIT
tokens followed by an optional DASH token etc etc...

- Kevin.