atoi/atof

[debjyoti]

Hi all

I was messing around with a perl script and realized converting
strings to integers/floats is iffy on perl (compared to C). There is
all this mention of atoi/atof being system dependent etc and so I
found this piece of code for atoi():

sub atoi {
my $t;
foreach my $d (split(//, shift())) {
$t = $t * 10 + $d;
}
return $t;
}

http://perl.plover.com/IAQ/IAQlist.html#how_do_i_convert_a_string_to_a_number

It worked for my applicaton and then I needed an atof(). It was easy
to enhance the above:

sub atof {
my $pre;
my $suf;
my $t;

$in = shift();
@dual = split(/\./,$in );
foreach my $d (split(//, $dual[0])) {
$pre = $pre * 10 + $d;
}
$div = 1;
foreach my $d (split(//, $dual[1])) {
$suf = $suf * 10 + $d;
$div *= 10;
}

$t = $pre + ($suf/$div);
#print "!!!debug!!! $in $dual[0] , $dual[1] [$pre + ($suf / $div) =
$t]\n";
return $t;
}

Putting it here incase it is useful to some perl newbie... and in case
I need it future

 

[Charlton Wilbur]

d> Hi all I was messing around with a perl script and realized
d> converting strings to integers/floats is iffy on perl (compared
d> to C).

String to integer:

my $out = int ($in);

String to float:

my $out = $in;

(Hint: the website you refer to -- http://perl.plover.com/IAQ/IAQlist.html
-- is not meant to be taken entirely seriously. Read some of the other
questions and answers if you doubt me.)

Charlton

 

[Jürgen Exner]

I was messing around with a perl script and realized converting
strings to integers/floats is iffy on perl (compared to C).

Really? I haven't noticed. And considering that Perl knows only scalars
(each of which has certain string, numerical, and boolean values) I
don't quite understand what problem you are tyring to solve.

There is
all this mention of atoi/atof being system dependent etc and so I
found this piece of code for atoi():

sub atoi {
my $t;
foreach my $d (split(//, shift())) {
$t = $t * 10 + $d;
}
return $t;
}

Is there anything wrong with just using $t in a numerical context, e.g.
adding 0?

It worked for my applicaton and then I needed an atof(). It was easy
to enhance the above:

Is there anything wrong with just using the scalar value in a numerical
context? (Well, I guess there could be due to the inherent problems of
floating point arithmetic)

Putting it here incase it is useful to some perl newbie... and in case
I need it future

Again, what problem exactly were you trying to solve where simply using
the scalar in the proper context didn't work?

jue

 

[sln]

Hi all

I was messing around with a perl script and realized converting
strings to integers/floats is iffy on perl (compared to C). There is
all this mention of atoi/atof being system dependent etc and so I
found this piece of code for atoi():

sub atoi {
my $t;
foreach my $d (split(//, shift())) {

Split() is being passed a string. So $d is now used as a string.
$t here is unknown at this point.

$t = $t * 10 + $d;

^ ^
Now $t and $d are being used as a number because of the *+ operators.
If $d were not a digit, there might be a runtime error.

}
return $t;
}

[snip]

Perl's got a kind of general purpose type SCALAR that interprets
if a number, string based on the usage of operators. Its interchangeable.
So its not really necessary to do this.

Of more use would be rounding in the conversion of float to int (both SCALARS).
Here you have to do it yourself. ie:
$needwholenumberrounded = int($floatnumber+.5);

In your examples you emulate the algorithym for the C atoi().
When in effect, Perl, built on C, calls atoi() directly from the standard runtime
libraries. So there's no need to duplicate the work.

I noticed in your emulation you don't account for [whitespaces][sign][digits].

Typically, atoi() from C library does something like this:

int _myatoi (char *p)
{
int num = 0, sign = 1, digit;
while (p && (digit=*p) == ' ' || digit == '\t') p++;
if (*p == '-') {
sign = -1;
p++;
} else
if (*p == '+')
p++;
while (digit = *p++) {
if (!(digit >= '0' && digit <= '9'))
return 0;
num = num * 10 + (digit - '0');
}
return sign*num;
}

Good job though, I like your emulations.


sln

 

[Peter J. Holzer]

Really?

Yes, really. The Perl routines for converting between strings and
numbers are buggy.

I haven't noticed.

I guess most people haven't, or the bugs would be fixed already. These
bugs have been there for a very long time, but they show up only if you
get close to the 15 digits/53 bits of FP precision.

I'm almost sure the OP's code is even buggier, though. String/number
conversions aren't trivial.

hp

 

[sln]

Hi all

I was messing around with a perl script and realized converting
strings to integers/floats is iffy on perl (compared to C). There is
all this mention of atoi/atof being system dependent etc and so I
found this piece of code for atoi():

sub atoi {
my $t;
foreach my $d (split(//, shift())) {

Split() is being passed a string. So $d is now used as a string.
$t here is unknown at this point.

$t = $t * 10 + $d;

^ ^
Now $t and $d are being used as a number because of the *+ operators.
If $d were not a digit, there might be a runtime error.

}
return $t;
}

[snip]

[snip]
I might as well be correct

Typically, atoi() from C library does something like this:

int _myatoi (char *p)
{
int num = 0, sign = 1, digit;

// assert(p);
if (!p) return 0;
while ((digit=*p)&& (digit == ' ' || digit == '\t')) p++;
if (digit == '-') {
sign = -1;
p++;
} else
if (digit == '+')
p++;

 

[sln]

Yes, really. The Perl routines for converting between strings and
numbers are buggy.


I guess most people haven't, or the bugs would be fixed already. These
bugs have been there for a very long time, but they show up only if you
get close to the 15 digits/53 bits of FP precision.

I'm almost sure the OP's code is even buggier, though. String/number
conversions aren't trivial.

hp

">>>strings to integers/floats is iffy on perl (compared to C). "

Aren't trivial? So ANSI C++ hasn't got it right yet? Perl had to bypass.
Its the other way around, number to string isin't trivial.


sln

 

[Ilya Zakharevich]

[A complimentary Cc of this posting was sent to

">>>strings to integers/floats is iffy on perl (compared to C). "

Aren't trivial?
Yes.

So ANSI C++ hasn't got it right yet?

What has C++ got to do with it? (And which particular runtime library
would you mean anyway? AFAIK, ANSI does not produce any compiler...)

Perl had to bypass.

For some unfathomable reasons, Perl (starting from about 5.8.1?)
bypasses CRTL routines...

Its the other way around, number to string isin't trivial.

??? Do not know what makes you think so. One is decimal-to-binary,
another is binary-to-decimal; both require higher precision (IIRC, 3
extra binary digits) than what the target conversion format can hold...

Yours,
Ilya

 

[Peter J. Holzer]

">>>strings to integers/floats is iffy on perl (compared to C). "

Aren't trivial? So ANSI C++ hasn't got it right yet?

"ANSI C++" is a specification. It only defines what needs to be done,
not how to do it. You would have to look at specific implementations.
Anyway, C++ is irrelevant here, as perl is written in C, not C++.

Perl had to bypass.

ANSI C has a strtod function for string -> floating point conversion.
I'm not sure if perl uses it (it's been some time since I looked at the
code, sorry), but it probably could and should use it, as I would expect
it to be correct on all major platforms. If perl doesn't use it, it is
probably for historical reasons. Easy to fix.

Unfortunately, there is no reverse function in C: sprintf generates a
fixed number of digits after the decimal point, which is either too
little or too much. So that's something perl has to do itself.

Its the other way around, number to string isin't trivial.

Conversion to and from integers is trivial (typical exercise for first
semester students). Conversion to and from FP numbers isn't trivial in
either direction. The situation is only asymmetric because C already has
a nice function for conversion from strings to numbers (which shields
the C programmer from all the hairy stuff), but not for the
reverse.

hp

 

[Ilya Zakharevich]

[A complimentary Cc of this posting was NOT [per weedlist] sent to
Peter J. Holzer

ANSI C has a strtod function for string -> floating point conversion.
I'm not sure if perl uses it (it's been some time since I looked at the
code, sorry), but it probably could and should use it, as I would expect
it to be correct on all major platforms. If perl doesn't use it, it is
probably for historical reasons. Easy to fix.

In fact, this particular misfortune is quite recent. Somebody did not
realize that the question is tricky, and inserted a wrong, homegrown,
implementation...

Unfortunately, there is no reverse function in C: sprintf generates a
fixed number of digits after the decimal point, which is either too
little or too much. So that's something perl has to do itself.

Nope, Perl's semantic coincides (unfortunately!) with sprintf()'s
one. It may have its own implementation of sprintf(), but no more
than that...

The alternative semantic (emit as few digits as the string==>number
function requires to reproduce the same number) is computationally
intensive. When, in 90's, a replacement was discussed, it led to
significant slowdowns. However, nobody at the moment realized that
Perl was not caching N-->S conversion agressively enough, so it was
repeating the conversion again and again. When later I found/fixed
the caching problem, the benchmarks were not (AFAIK) redone..

The third problem is that one wants to minimize loads on
newsgroup/mailing-list "why numeric calculations are so weird in
Perl"; so the precision of N-->S is deliberately chosen so low, that
indications of weirdness of FP arithmetic are not manifested as often
as it would be with "precise" algorithms...

Hope this helps,
Ilya

 

[sln]

"ANSI C++" is a specification. It only defines what needs to be done,
not how to do it. You would have to look at specific implementations.
Anyway, C++ is irrelevant here, as perl is written in C, not C++.


ANSI C has a strtod function for string -> floating point conversion.
I'm not sure if perl uses it (it's been some time since I looked at the
code, sorry), but it probably could and should use it, as I would expect
it to be correct on all major platforms. If perl doesn't use it, it is
probably for historical reasons. Easy to fix.

Unfortunately, there is no reverse function in C: sprintf generates a
fixed number of digits after the decimal point, which is either too
little or too much. So that's something perl has to do itself.


Conversion to and from integers is trivial (typical exercise for first
semester students). Conversion to and from FP numbers isn't trivial in
either direction. The situation is only asymmetric because C already has
a nice function for conversion from strings to numbers (which shields
the C programmer from all the hairy stuff), but not for the
reverse.

hp

Well, as you said ANSI is just a specification. But its specific enough.
I'm going out on a limb here and listing a specific implementation for
Microsofts C libraries, ie: all Window's platforms:

ANSI Windows
String->Number Number->String
----------------- ----------------
atoi/atol/strol _itoa/_ltoa
atof/strtod _fcvt/_ecvt


I think these are C. Seems unusual Perl is built on
platforms without using thier ANSI and/or OS specific functions.

A large majority of these conversions are done in assembly
(not just compiled into assembly) and fully optimized.

I can see possibly not wanting to deal with the individual
platrom exception processing though.

It just never occurred to me these standard calls are not made,
and home-grown routines are used instead. Its got to be slower.

Apparently there is no ASNI equivalent Number->String. Most
platforms have them though.

String->Number is a one way street to load parameters.
Number->String is a view.
There should be no circular input/output relationship between them.

It's been my experience with C/C++ that precision is the culprit
and cause of calculation "drift" and/or unexpected results.
Nothing too real is measurable/manufacturable below 1/10 of a micron.
Thats not alot of decimal places. As soon as a term above that is introduced
into a calculation, terms below are insignificant.

Therefore IMO, PI = 3.14


sln

 

[Peter J. Holzer]

Nope, Perl's semantic coincides (unfortunately!) with sprintf()'s
one. It may have its own implementation of sprintf(), but no more
than that...

I may be overlooking something obvious, but which format option will
produce the same output as perl's float->string conversion?

The third problem is that one wants to minimize loads on
newsgroup/mailing-list "why numeric calculations are so weird in
Perl"; so the precision of N-->S is deliberately chosen so low, that
indications of weirdness of FP arithmetic are not manifested as often
as it would be with "precise" algorithms...

Argh! I've had that discussion with HP several years ago. Fortunately
they conceded the point that FORTRAN should not deliberatly introduce
errors to cater to the expectations of naive users. I don't think perl
should either.

hp

 

[Peter J. Holzer]

ANSI C has a strtod function for string -> floating point conversion. [...]
Unfortunately, there is no reverse function in C: sprintf generates a
fixed number of digits after the decimal point, which is either too
little or too much. So that's something perl has to do itself.

Its the other way around, number to string isin't trivial.

Conversion to and from integers is trivial (typical exercise for first
semester students). Conversion to and from FP numbers isn't trivial in
either direction. The situation is only asymmetric because C already has
a nice function for conversion from strings to numbers (which shields
the C programmer from all the hairy stuff), but not for the
reverse.

Well, as you said ANSI is just a specification. But its specific enough.

It doesn't give any guarantees about precision for FP operations. An
implementation may get strtod quite wrong and still be compliant.

I'm going out on a limb here and listing a specific implementation for
Microsofts C libraries, ie: all Window's platforms:

ANSI Windows
String->Number Number->String
----------------- ----------------
atoi/atol/strol _itoa/_ltoa
atof/strtod _fcvt/_ecvt


I think these are C. Seems unusual Perl is built on
platforms without using thier ANSI and/or OS specific functions.

man fcvt:

| NOTES
| These functions are obsolete. Instead, sprintf(3) is recommended.

AFAICS, ecvt and fcvt don't give you anything sprintf doesn't.

It just never occurred to me these standard calls are not made,
and home-grown routines are used instead. Its got to be slower.

Speed is indeed a problem in perl, which does a lot of implicit
conversion from string to number and back. But lack of accuracy is IMHO
the bigger problem.

Apparently there is no ASNI equivalent Number->String. Most
platforms have them though.

String->Number is a one way street to load parameters.
Number->String is a view.
There should be no circular input/output relationship between them.

On the contrary. There must be a circular relationship between them, if
you want to be able to write numbers to a file (or send them via the
network) and read them back. And for perl those conversions even occur
internally, so they must be reversible.

hp

 

[Ilya Zakharevich]

[A complimentary Cc of this posting was NOT [per weedlist] sent to
Peter J. Holzer

I may be overlooking something obvious, but which format option will
produce the same output as perl's float->string conversion?

Probably %g with precision governed by $#, IIRC.

Argh! I've had that discussion with HP several years ago. Fortunately
they conceded the point that FORTRAN should not deliberatly introduce
errors to cater to the expectations of naive users. I don't think perl
should either.

10 years ago the newsgroup was quite overflown by messages like this.
I agree that the situation today is significantly different, so the
old preferences might became easier to fight with...

Yours,
Ilya

 

[Peter J. Holzer]

[A complimentary Cc of this posting was NOT [per weedlist] sent to
Peter J. Holzer

I may be overlooking something obvious, but which format option will
produce the same output as perl's float->string conversion?

Probably %g with precision governed by $#, IIRC.

I thought %.15g would fill with zeros, like %.15f does, but it doesn't.
My C is obviously getting rusty :-(.

So, yes, you're right. Perl mimics sprintf.

hp