J
Jacek Dziedzic
Hello!
This is my first time dealing with Very Large Files.
I have vector of strings representing numbers and I need
to extract bytes in binary mode from a Large File that
correspond to ranges specified by the strings. For example
for an input of
"0", "100", "500", "700"
I need to create three files, the first would contain
bytes 0..99 of the original file, the second: bytes 100..499
of the original file and the third: bytes 500..699 of the
original file.
All is fine, except the original file is 14 GB long and
most of the offsets represent numbers that are larger
than numeric_limits<unsigned long int>::max() which is
2^32-1 on my system, since sizeof(unsigned long int)==4 here.
So it seems I cannot use any of the portable builtin integer
types to calculate offsets, right? Still, seekg() takes
an argument of type streampos, so I figure such large
files should be supported by my implementation <OT>(g++ 4.1.2)</OT>.
How does one use this streampos type? sizeof(streampos)==16
on my system, so at first I thought it is a Large Integer
that can hold numbers up to 2^128-1. I tried outputting
numeric_limits<streampos>::max() but it says 0.
I am now confused -- it this numeric limit not supported
for streampos or is it cout that is unable to deal with
a number that large?
I also noticed that arithmetic operations are not defined
for this type, at least ++ was not defined. How do I calculate
differences between file offsets stored in streampos then?
I tried this:
ifstream in("myfile");
in.seekg(0,ios_base::end);
streampos sp = in.tellg();
cout << sp << endl;
but this throws (!) an exception with what() of:
"basic_string::_S_construct NULL not valid".
Does that mean that cout cannot be used with this streampos type?
So, to summarize my doubts. How do I portably use values
stored in a streampos to calculate differences between offsets?
I would like to tell the user that "I am now extracting
bytes 7000000000..12000000000, a total of 5000000000 bytes".
I don't want to use any _int64 or such. How do I output these
large values to an ostream? How do I tell what is the largest
file size supported on my platform?
TIA,
- J.
This is my first time dealing with Very Large Files.
I have vector of strings representing numbers and I need
to extract bytes in binary mode from a Large File that
correspond to ranges specified by the strings. For example
for an input of
"0", "100", "500", "700"
I need to create three files, the first would contain
bytes 0..99 of the original file, the second: bytes 100..499
of the original file and the third: bytes 500..699 of the
original file.
All is fine, except the original file is 14 GB long and
most of the offsets represent numbers that are larger
than numeric_limits<unsigned long int>::max() which is
2^32-1 on my system, since sizeof(unsigned long int)==4 here.
So it seems I cannot use any of the portable builtin integer
types to calculate offsets, right? Still, seekg() takes
an argument of type streampos, so I figure such large
files should be supported by my implementation <OT>(g++ 4.1.2)</OT>.
How does one use this streampos type? sizeof(streampos)==16
on my system, so at first I thought it is a Large Integer
that can hold numbers up to 2^128-1. I tried outputting
numeric_limits<streampos>::max() but it says 0.
I am now confused -- it this numeric limit not supported
for streampos or is it cout that is unable to deal with
a number that large?
I also noticed that arithmetic operations are not defined
for this type, at least ++ was not defined. How do I calculate
differences between file offsets stored in streampos then?
I tried this:
ifstream in("myfile");
in.seekg(0,ios_base::end);
streampos sp = in.tellg();
cout << sp << endl;
but this throws (!) an exception with what() of:
"basic_string::_S_construct NULL not valid".
Does that mean that cout cannot be used with this streampos type?
So, to summarize my doubts. How do I portably use values
stored in a streampos to calculate differences between offsets?
I would like to tell the user that "I am now extracting
bytes 7000000000..12000000000, a total of 5000000000 bytes".
I don't want to use any _int64 or such. How do I output these
large values to an ostream? How do I tell what is the largest
file size supported on my platform?
TIA,
- J.