loss of precision in doing file I/O

[giff]

Hi all,

I have a doubt, I'll try to expose it to you as clearly as I can, maybe
it is not completely in topic, sorry about that.

I have a few vectors of doubles (output of some calculations) and I
check the norm and the dot product between couples of them before
storing them in a txt file, obtaining the expected results.

The vectors are orthonormal so the norm=1 and the dot product gives
something like 1.0e-18 that I suppose is quite close to zero...

When I read the vectors again from the txt file, I check the same things
and I find that the dot product is now around 1.0e-8.

Is this normal? Is there a way to avoid this loss of precision?

I am programming in c++ using visual studio 6 on win2000.

Thanks for your precious help.

 

[osmium]

I have a doubt, I'll try to expose it to you as clearly as I can, maybe it
is not completely in topic, sorry about that.

I have a few vectors of doubles (output of some calculations) and I check
the norm and the dot product between couples of them before storing them
in a txt file, obtaining the expected results.

The vectors are orthonormal so the norm=1 and the dot product gives
something like 1.0e-18 that I suppose is quite close to zero...

When I read the vectors again from the txt file, I check the same things
and I find that the dot product is now around 1.0e-8.

Is this normal? Is there a way to avoid this loss of precision?

I am programming in c++ using visual studio 6 on win2000.

If you read and write to the file using >> and << the data will be converted
to a character representation and the default precision will be used.
1.0e-8 sounds about like what I would expect. You could change the
precision [see ios:recision() ] or save the file in the binary form [see
read() and write()].

 

[mlimber]

Hi all,

I have a doubt, I'll try to expose it to you as clearly as I can, maybe
it is not completely in topic, sorry about that.

I have a few vectors of doubles (output of some calculations) and I
check the norm and the dot product between couples of them before
storing them in a txt file, obtaining the expected results.

The vectors are orthonormal so the norm=1 and the dot product gives
something like 1.0e-18 that I suppose is quite close to zero...

When I read the vectors again from the txt file, I check the same things
and I find that the dot product is now around 1.0e-8.

Is this normal? Is there a way to avoid this loss of precision?

See this FAQ and those following:

http://www.parashift.com/c++-faq-lite/newbie.html#faq-29.16

Can you show us a *complete* but *minimal* code snippet that we can cut
and paste into our editors to try it? It may be user error.

I am programming in c++ using visual studio 6 on win2000.

Irrelevant. (If it is relevant, you're in the wrong newsgroup.)

Cheers! --M

 

[Victor Bazarov]

I have a few vectors of doubles (output of some calculations) and I
check the norm and the dot product between couples of them before
storing them in a txt file, obtaining the expected results.

The vectors are orthonormal so the norm=1 and the dot product gives
something like 1.0e-18 that I suppose is quite close to zero...

When I read the vectors again from the txt file, I check the same
things and I find that the dot product is now around 1.0e-8.

Is this normal? Is there a way to avoid this loss of precision?

Write more digits to the output.

I am programming in c++ using visual studio 6 on win2000.

That shoulnd't matter.

V

 

[giff]

osmium ha scritto:

If you read and write to the file using >> and << the data will be converted
to a character representation and the default precision will be used.

that's probably what happens

1.0e-8 sounds about like what I would expect. You could change the
precision [see ios:recision() ] or save the file in the binary form [see
read() and write()].

thanks for the hint

 

[giff]

mlimber ha scritto:

See this FAQ and those following:

http://www.parashift.com/c++-faq-lite/newbie.html#faq-29.16
thanks


Can you show us a *complete* but *minimal* code snippet that we can cut
and paste into our editors to try it? It may be user error.

std:fstream os("filename.txt" , ios_base::trunc|ios_base:ut );

for(int n=0; n<nP; n++){

for(int i=0; i<nV; i++){

os << ((double*)(shapeV2d[n]->data.ptr))[i * 2] << " "
<< ((double*)(shapeV2d[n]->data.ptr))[i*2+1] << endl;

}
}

I write nP vectors stored as CvMat arrays (I am using the opencv libraries)

here is the reading routine:

std::ifstream is( filename.txt );

for(int n=0; n<nP; n++ ) {

for(int v=0; v<nV; v++){

is >> x >> y;
((double*)(shapeV2d[n]->data.ptr))[2 * v] = x;
((double*)(shapeV2d[n]->data.ptr))[2*v+1] = y;

}
}


Anyway I don't think this could be a huge problem for my project, I mean
1.0e-8 is still quite small and comparable to zero...

thanks

 

[Marcus Kwok]

I have a few vectors of doubles (output of some calculations) and I
check the norm and the dot product between couples of them before
storing them in a txt file, obtaining the expected results.

The vectors are orthonormal so the norm=1 and the dot product gives
something like 1.0e-18 that I suppose is quite close to zero...

When I read the vectors again from the txt file, I check the same things
and I find that the dot product is now around 1.0e-8.

I ran into something similar before where I would take a dot product of
a couple different (mathematical) vectors and then normalize them. When
the dot product was 1e-18 or whatever, the algorithm normalized this
which obviously threw all my calculations out the window. I had to
explicitly set the result to 0 in order for things to work correctly.

For example,

const double Epsilon = 1e-10; // or whatever is appropriate

// ...

double result = /* compute dot product */;

// need #include <cmath> for std::abs
if (std::abs(result) < Epsilon) {
result = 0;
}

Is this normal? Is there a way to avoid this loss of precision?

The above are symptoms of the limitations of floating-point type numbers
in general. You can search the FAQ for more info. In particular, the
FAQ has a link to David Goldberg's paper "What Every Computer-Scientist
Should Know About Floating Point Arithmetic".

 

[giff]

Victor Bazarov ha scritto:

Write more digits to the output.

I am using <<, do you mean I should use a printf instead?

 

[Victor Bazarov]

Victor Bazarov ha scritto:


I am using <<, do you mean I should use a printf instead?

No, I don't mean that. RTFM about stream manipulators and
specifically about 'setprecision'.

V

 

[Jeff Flinn]

Write more digits to the output.


That shoulnd't matter.

Aah, but it does. There have been recent discussions on the boost mailing
list concerning roundtripping doubles/floats to/from text files with many VC
versions losing 1-2 bits precision for certain numeric values. These have
been posted on MS website as bugs, but have apparently been refuted by MS as
a design feature.

Jeff Flinn

 

[osmium]

Aah, but it does. There have been recent discussions on the boost mailing
list concerning roundtripping doubles/floats to/from text files with many
VC versions losing 1-2 bits precision for certain numeric values. These
have been posted on MS website as bugs, but have apparently been refuted
by MS as a design feature.

Look at the numbers the OP gave! This thread is not about losing a few bits
between friends.

 

[Jeff Flinn]

Look at the numbers the OP gave! This thread is not about losing a
few bits between friends.

Which is why I responded only to Victor's second point. Victor addressed the
large scale disparity in his first response. Once that is corrected, the OP
and possibly others Googling this thread can be saved some time in tracking
down these finer grain disparities.

Jeff Flinn

 

[Victor Bazarov]

Which is why I responded only to Victor's second point. Victor
addressed the large scale disparity in his first response. Once that
is corrected, the OP and possibly others Googling this thread can be
saved some time in tracking down these finer grain disparities.

I don't understand this. You "addressed" my second point by plucking
it out of context altogether. WTF?

V

 

[Jeff Flinn]

I don't understand this. You "addressed" my second point by plucking
it out of context altogether. WTF?

I didn't do the plucking. The context disappeared in osmium's reponse.

Jeff

 

[mlimber]

mlimber ha scritto:

[snip code]

Not quite what was requested since we couldn't cut, paste, and run that
without manufacturing a lot of other stuff (e.g., the ShapeV2d
structure type, main(), header files, etc.), but on inspection, it
certainly seems likely that, as others suggested, it is a problem with
the digits of precision you are writing to the file.

Cheers! --M

 

[Giff]

mlimber ha scritto:

Not quite what was requested since we couldn't cut, paste, and run that
without manufacturing a lot of other stuff

mmm right, sorry about that...