File Size Limit Exceeded - How to handle work around this?

[eastcoastguyz]

I wrote a simple program to continue to create a very large file (on
purpose), and even though there is plenty of disk space on that device
the program aborted with the error message "File Size Limit Exceeded".
The file size was 2147483647. I checked ulimit -a and its set to
unlimited.

Is this a compiler issue? I would like to see a C code example of how
to increase the limit or make it unlimited (if that is a wise thing to
do).

Thanks in advance!

 

[Walter Roberson]

I wrote a simple program to continue to create a very large file (on
purpose), and even though there is plenty of disk space on that device
the program aborted with the error message "File Size Limit Exceeded".
The file size was 2147483647. I checked ulimit -a and its set to
unlimited.

Is this a compiler issue?

Probably not.

I would like to see a C code example of how
to increase the limit or make it unlimited (if that is a wise thing to
do).

It is probably an operating system limitation (or a disk quota
limitation). ulimit (which is not part of C) with -a set to
unlimited, just means that the operating system will allow you to
write files as big as is supported by that particular file system.

Whether your operating system supports larger files at all would
be OS specific, as would be any special means to create such files.
This is a matter that should be taken to a resource that deals
with your specific OS. It is -possible- that they will tell you
there to change some flags to your compiles, but that would be
for deep OS implementation reasons, not for reasons directly related
to standard C.

 

[dcorbit]

I wrote a simple program to continue to create a very large file (on
purpose), and even though there is plenty of disk space on that device
the program aborted with the error message "File Size Limit Exceeded".
The file size was 2147483647. I checked ulimit -a and its set to
unlimited.

Is this a compiler issue? I would like to see a C code example of how
to increase the limit or make it unlimited (if that is a wise thing to
do).

Thanks in advance!

Write the file in blocks and maintain your own chain or use a database,
which will already have done that

 

[eastcoastguyz]

I forgot to mention, the OS is Linux with the latest version of CentOS
using 'cc'.

 

[Michael]

I wrote a simple program to continue to create a very large file (on

purpose), and even though there is plenty of disk space on that device
the program aborted with the error message "File Size Limit Exceeded".
The file size was 2147483647. I checked ulimit -a and its set to
unlimited.

Is this a compiler issue? I would like to see a C code example of how
to increase the limit or make it unlimited (if that is a wise thing to
do).

It's been a long time, though, so this advice may be dated.

A long time ago, when the Linux 2.4 kernel was new (i.e., when 64-bit
file sizes were just starting to catch on), we used to have this same
problem. Back then, you had to compile with some special symbol
defined, LARGE_FILES or something like that. That caused the typical
file size types (offset_t and the like) to be 64 bit. You might find
something like that in the compiler documentation.

Of course, to narrow it to OS or C, you could have your program write
to stdout and redirect to a file. If it still doesn't work, you have
an OS problem. And if it isn't C, it could be your OS kernel, or your
shell, or your filesystem (e.g., NFS) that has some kind of limit in
it.

Michael

 

[eastcoastguyz]

It's been a long time, though, so this advice may be dated.

A long time ago, when the Linux 2.4 kernel was new (i.e., when 64-bit
file sizes were just starting to catch on), we used to have this same
problem. Back then, you had to compile with some special symbol
defined, LARGE_FILES or something like that. That caused the typical
file size types (offset_t and the like) to be 64 bit. You might find
something like that in the compiler documentation.

Of course, to narrow it to OS or C, you could have your program write
to stdout and redirect to a file. If it still doesn't work, you have
an OS problem. And if it isn't C, it could be your OS kernel, or your
shell, or your filesystem (e.g., NFS) that has some kind of limit in
it.

Michael

Thanks for the suggestion. I made another version of this C program,
where it writes to standard output and piped the output to a file. It
had no problem creating a file size over 3GB, exceeding the 2GB limit I
encountered.
This system is using gcc version 3.4.6 20060404 (Red Hat 3.4.6-3) which
is what came with the latest version of CentOS. The kernel is:
[root@localhost proc]# cat /proc/version
Linux version 2.6.9-42.0.3.EL (buildsvn@build-i386) (gcc version 3.4.6
20060404

 

[loic-dev]

Hello,

Try to compile your program with the following flags:
-D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS 64

HTH,
Loic.

 

[Kohn Emil Dan]

Hello,


Try to compile your program with the following flags:
-D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS 64

Probably he meant:

-D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64

 

[eastcoastguyz]

Probably he meant:

-D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64

Yes! This worked. Thanks to everyone who posted in response to my
request for help.

Does anyone know if it is required to do these compiler options
regardless of the hardware the same OS runs on? Thanks!

 

[Nelu]

Yes! This worked. Thanks to everyone who posted in response to my
request for help.

Does anyone know if it is required to do these compiler options
regardless of the hardware the same OS runs on? Thanks!

<OT>
I think the only macro you need defined is _GNU_SOURCE which will
enable a number of macros and the ...LARGEFILE64... ones are
among them. I think this is the preferred way to do this. I
couldn't find _GNU_SOURCE in my header files on 64 bit Gentoo
</OT>
but it may be because I didn't pay too much attention. You
shouldn't have posted the question here and you should not
consider advices received from this group. The standard C doesn't
know about 64 vs 32 bit, files larger than 2 GB or specific
extensions. This is a question for a group that deals with the
combination of compiler/OS that you're using.