read 1k * 1k data

[a]

To solve the 100k * 100k data, I finally adopt the file read method and use
malloc. The system somehow knows to use virtual memory.

Now I first test 1k * 1k data but something uninterpretable happens again.

The data file starts with:

42.106983132 85.931514337 98.155213938 23.685776453 76.827067592
....

when I print out the array storing the file content:

0.000000 k: 1020 r: 0.000000 k: 1021 r: 0.000000 k: 1022 r: 0.000000 k: 1023
r: 0.000000 k: 1024 r: 42.106983 k: 1025 r: 85.931514 k:

one can see the first line reading nothing (k as index and r as array
content)

I use the function fscanf to read the file:

#define SIZE 1024

double *r;
int i = 0;

r = (double *)malloc(SIZE * SIZE * sizeof(double));


while(!feof(file) && i<SIZE * SIZE) {
fscanf(file, "%lf", &r);
i++;

// if (i < 30 ) return;
}
for (k=0; k<SIZE*SIZE; k++) {
// if (k> SIZE * SIZE - 10 )
if (k< 2048 )
printf("k: %d r: %lf ",k, r[k]);
}


and indeed if (i < 30) return does not stop the file reading loop and i need
to add i < SIZE * SIZE to stop infinite reading. Anybody knows what's
happening?

Furthermore, when I use scanf, fscanf etc to search in Google, it returns
not so helpful example results. I'd also appreciate your suggestions on
searching help on web. Thanks a lot!!

 

[Eric Sosman]

To solve the 100k * 100k data, I finally adopt the file read method and use
malloc. The system somehow knows to use virtual memory.

Now I first test 1k * 1k data but something uninterpretable happens again.

The data file starts with:

42.106983132 85.931514337 98.155213938 23.685776453 76.827067592
...

when I print out the array storing the file content:

0.000000 k: 1020 r: 0.000000 k: 1021 r: 0.000000 k: 1022 r: 0.000000 k: 1023
r: 0.000000 k: 1024 r: 42.106983 k: 1025 r: 85.931514 k:

one can see the first line reading nothing (k as index and r as array
content)

I use the function fscanf to read the file:

#define SIZE 1024

double *r;
int i = 0;

r = (double *)malloc(SIZE * SIZE * sizeof(double));


while(!feof(file) && i<SIZE * SIZE) {

This mistake is the subject of Question 12.2 in the
comp.lang.c Frequently Asked Questions (FAQ) list,

fscanf(file, "%lf", &r);

Do you have any reason to believe that this worked?
Question 12.19 doesn't address this issue directly, but
has a few hints about it.

i++;

// if (i < 30 ) return;
}
for (k=0; k<SIZE*SIZE; k++) {
// if (k> SIZE * SIZE - 10 )
if (k< 2048 )
printf("k: %d r: %lf ",k, r[k]);

This mistake is the subject of Question 12.9.

 

[Ben Pfaff]

#define SIZE 1024

double *r;
int i = 0;

r = (double *)malloc(SIZE * SIZE * sizeof(double));

The most likely result of this size calculation is 8388608. Are
you on a system where `int' can hold this value? `int' may have
a maximum value as small as 32767.

I don't recommend casting the return value of malloc():

* The cast is not required in ANSI C.

* Casting its return value can mask a failure to #include
<stdlib.h>, which leads to undefined behavior.

* If you cast to the wrong type by accident, odd failures can
result.

In unusual circumstances it may make sense to cast the return value of
malloc(). P. J. Plauger, for example, has good reasons to want his
code to compile as both C and C++, and C++ requires the cast, as he
explained in article <[email protected]>.
However, Plauger's case is rare indeed. Most programmers should write
their code as either C or C++, not in the intersection of the two.

When calling malloc(), I recommend using the sizeof operator on
the object you are allocating, not on the type. For instance,
*don't* write this:

int *x = malloc (128 * sizeof (int)); /* Don't do this! */

Instead, write it this way:

int *x = malloc (128 * sizeof *x);

There's a few reasons to do it this way:

* If you ever change the type that `x' points to, it's not
necessary to change the malloc() call as well.

This is more of a problem in a large program, but it's still
convenient in a small one.

* Taking the size of an object makes writing the statement
less error-prone. You can verify that the sizeof syntax is
correct without having to look at the declaration.

 

[Malcolm McLean]

To solve the 100k * 100k data, I finally adopt the file read method and
use malloc. The system somehow knows to use virtual memory.

Now I first test 1k * 1k data but something uninterpretable happens again.

The data file starts with:

42.106983132 85.931514337 98.155213938 23.685776453
76.827067592 ...

when I print out the array storing the file content:

0.000000 k: 1020 r: 0.000000 k: 1021 r: 0.000000 k: 1022 r: 0.000000 k:
1023 r: 0.000000 k: 1024 r: 42.106983 k: 1025 r: 85.931514 k:

one can see the first line reading nothing (k as index and r as array
content)

I use the function fscanf to read the file:

#define SIZE 1024

double *r;
int i = 0;

r = (double *)malloc(SIZE * SIZE * sizeof(double));


while(!feof(file) && i<SIZE * SIZE) {
fscanf(file, "%lf", &r);
i++;

// if (i < 30 ) return;
}
for (k=0; k<SIZE*SIZE; k++) {
// if (k> SIZE * SIZE - 10 )
if (k< 2048 )
printf("k: %d r: %lf ",k, r[k]);
}


and indeed if (i < 30) return does not stop the file reading loop and i
need to add i < SIZE * SIZE to stop infinite reading. Anybody knows what's
happening?

Furthermore, when I use scanf, fscanf etc to search in Google, it returns
not so helpful example results. I'd also appreciate your suggestions on
searching help on web. Thanks a lot!!

This looks OK to me.
Try putting in a test after malloc() to make sure you actually have the
memory.
Also put a newline on the end of your diagnostic printf().
Then
1) post the code where you open the file
2) try echoing the file characters back to stdout to make sure the file is
being read correctly. You will need to use fgetc().
3) Try reading to a temporary double and printing back to make sure sfcanf()
is converting the information properly.

 

[Old Wolf]

The most likely result of this size calculation is 8388608. Are
you on a system where `int' can hold this value? `int' may have
a maximum value as small as 32767.

sizeof(double) gives a size_t, so the int that is SIZE*SIZE
= 1048576 will be promoted to size_t for the multiplication.

 

[CBFalconer]

sizeof(double) gives a size_t, so the int that is SIZE*SIZE
= 1048576 will be promoted to size_t for the multiplication.

No cast needed. However the SIZE * SIZE calculation may be
performed before multiplication by sizeof(double), and that
preliminary may overflow (ints do not have to extend past 32767)
and cause un (or implementation) defined behavior.

 

[Charlie Gordon]

No cast needed. However the SIZE * SIZE calculation may be
performed before multiplication by sizeof(double), and that
preliminary may overflow (ints do not have to extend past 32767)
and cause un (or implementation) defined behavior.

More precisely: SIZE * SIZE shall be performed before multiplication by
sizeof(double) because multiplication is left to right associative. If
MAX_INT is 32767, SIZE * SIZE will overflow, even if size_t happens to be a
larger type.

 

[Charlie Gordon]

sizeof(double) gives a size_t, so the int that is SIZE*SIZE
= 1048576 will be promoted to size_t for the multiplication.

Almost true: the int that is SIZE * SIZE may have overflowed, then the
result will be promoted to size_t before multiplication by sizeof(double).
Unless size_t is a type smaller than int, in which case the whole expression
is evaluated as int and is signed (!).

 

[santosh]

More precisely: SIZE * SIZE shall be performed before multiplication
by sizeof(double) because multiplication is left to right associative.
If MAX_INT is 32767, SIZE * SIZE will overflow, even if size_t happens
to be a larger type.

Couldn't a smart compiler note that the final value must be of type
size_t and hence perform all operations with that type, thus avoiding
the potential int overflow?

 

[Charlie Gordon]

Couldn't a smart compiler note that the final value must be of type
size_t and hence perform all operations with that type, thus avoiding
the potential int overflow?

It would not be smart, it would be non conforming.
A smart compiler will detect the overflow in the constant expression and
report it to the programmer with a warning. That is if the programmer is
smart enough to ask for such reports ;-)

 

[Ben Pfaff]

It would not be smart, it would be non conforming.

Not true: SIZE * SIZE is the product of two `int's. If it
overflows, then behavior is undefined. Thus, the compiler can do
whatever it likes in that case, including evaluate the expression
as type size_t.

 

[Malcolm McLean]

Not true: SIZE * SIZE is the product of two `int's. If it
overflows, then behavior is undefined. Thus, the compiler can do
whatever it likes in that case, including evaluate the expression
as type size_t.

Whilst if all operands are size_t the behaviour is defined. Therefore the
program cannot print out an error message. However this is a hollow victory
indeed, because the overflow will not do what you want.

 

[CBFalconer]

"CBFalconer" <[email protected]> a écrit:
.... snip ...

More precisely: SIZE * SIZE shall be performed before
multiplication by sizeof(double) because multiplication is left
to right associative. If MAX_INT is 32767, SIZE * SIZE will
overflow, even if size_t happens to be a larger type.

No, less precisely. There is no restriction on the order of
operations. See the standard. This MAY happen, not WILL happen.

 

[CBFalconer]

Almost true: the int that is SIZE * SIZE may have overflowed, then
the result will be promoted to size_t before multiplication by
sizeof(double). Unless size_t is a type smaller than int, in which
case the whole expression is evaluated as int and is signed (!).

No. Once the int overflow occurs things are undefined.

 

[James Kuyper]

No, less precisely. There is no restriction on the order of
operations. See the standard. This MAY happen, not WILL happen.

There is no direct restriction on the order; but there are dependencies.
You need to know the value of the left operand of the second
multiplication before you can calculate the value resulting from that
multiplication. Since the left operand is itself a multiplication
operation, the first multiplication has to be complete before the second
one can be carried out.

That value dependency is all that's needed to realize that a signed
integer overflow is possible, depending only upon the value of SIZE.

 

[Dik T. Winter]

>
> There is no direct restriction on the order; but there are dependencies.

It is slightly different. The standard specifies a order of the operations
(using the syntax rules), and type conversions apply to *that* order. The
standard also allows change of order if for well-defined expressions the
result does not change with that change of order. In this case SIZE * SIZE
(the first expression to consider) will overflow, so from that point the
result is undefined. The compiler may change the order, but with respect
to the standard the result is still undefined.