[C99] local VLA with dimension given by a global var?

M

MackS

The system I am working on supports a subset of C99, among which
"standard-compliant VLAs".

I've already learnt that VLAs can't have global scope. My question is
whether I can safely declare a (local) VLA to have as its dimension a
global variable of type int:

short dim;

int main(void)
{

/* (...), possibly access/change value of dim */

{
int vla[dim];
}

/* (...) */

return 0;

}

If I understand the vary basics of how the compiler works this is a
trivial question, ie, in the code above the variable dim is just being
evaluated at that point and its value *at that point in time* is being
used to define the dimension of the array; is this correct? Ie, can I
safely this?

Thank you,

Mack
 
J

j

MackS said:
The system I am working on supports a subset of C99, among which
"standard-compliant VLAs".

I've already learnt that VLAs can't have global scope. My question is
whether I can safely declare a (local) VLA to have as its dimension a
global variable of type int:

short dim;

int main(void)
{

/* (...), possibly access/change value of dim */

{
int vla[dim];
}

/* (...) */

return 0;

}

If I understand the vary basics of how the compiler works this is a
trivial question, ie, in the code above the variable dim is just being
evaluated at that point and its value *at that point in time* is being
used to define the dimension of the array; is this correct? Ie, can I
safely this?

Thank you,

Mack

Yes, this is perfectly fine. 6.7.5.2#9 illustrates this.
 
J

Jack Klein

The system I am working on supports a subset of C99, among which
"standard-compliant VLAs".

I've already learnt that VLAs can't have global scope. My question is

It has nothing to do with scope, it has to do with storage duration.
VLAs must have automatic storage duration, and can't have static
storage duration.
whether I can safely declare a (local) VLA to have as its dimension a
global variable of type int:

short dim;

int main(void)
{

/* (...), possibly access/change value of dim */

{
int vla[dim];
}

/* (...) */

return 0;

}

If I understand the vary basics of how the compiler works this is a
trivial question, ie, in the code above the variable dim is just being
evaluated at that point and its value *at that point in time* is being
used to define the dimension of the array; is this correct? Ie, can I
safely this?

Yes, the size of the VLA will be the value that the expression
evaluates to at the time the array is created.
 
Z

Zeljko Vrba

It has nothing to do with scope, it has to do with storage duration.
VLAs must have automatic storage duration, and can't have static
storage duration.
Speaking of VLAs..

It seems that the standard doesn't specify how VLAs are implemented, although
the most obvious ways are:
1. stack (like the non-portable alloca())
2. malloc() on the heap

In both cases you can declare an absurdly large VLA so that there isn't enough
memory to allocate it. I haven't been able to find in the C99 standard what
happens then..

So what is the 'standard' way to handle an out-of-memory condition for VLAs?
There is no place to 'return NULL' as malloc() does..
 
M

Michael Mair

Zeljko said:
Speaking of VLAs..

It seems that the standard doesn't specify how VLAs are implemented, although
the most obvious ways are:
1. stack (like the non-portable alloca())
2. malloc() on the heap

In both cases you can declare an absurdly large VLA so that there isn't enough
memory to allocate it. I haven't been able to find in the C99 standard what
happens then..

So what is the 'standard' way to handle an out-of-memory condition for VLAs?
There is no place to 'return NULL' as malloc() does..

As this is not your problem but the implementation's:
The program will die. If you are lucky, it will do so in a verbose
way. I would not nurse any expectations beyond a size of 64K...

Cheers
Michael
 
Z

Zeljko Vrba

As this is not your problem but the implementation's:
The program will die. If you are lucky, it will do so in a verbose
way. I would not nurse any expectations beyond a size of 64K...
OK, I'm not trying to start a flame here, but why would anyone use VLAs when
they are not guaranteed to always work and you cannot detect that? In all other
aspects of C the programmer is given a chance to handle the error (extending
your line of thought - why check for malloc() returning NULL? the library coud
just terminate the program..)

I think this is a crucial point missing from the standard. The standard
commitee should have taken this into account and specify some kind of
standard behaviour that is under programmer's control (e.g. delivery of a
signal for which a signal handler can be written, at least in the hosted
environments). Otherwise, each implementation will choose its own way of
letting the programmer know about the memory allocation failure (e.g.
#pragma VLA_alloc_failure(handler_func) and similar..) and soon there could
be several incompatible implementations and the C99 code which uses custom VLA
error handling as an extension won't be portable accross them any more..

What's worse, the implementation can *never* guarantee ANY size of VLA because
it depends on the current runtime environment (e.g. in one environment I could
possibly get 16M VLA, and in the other I couldn't get even 64k).

You can't even know in advance the maximum amount of memory needed for VLAs
so it could be preallocated at program startup (or the execution fails if it
can't be preallocated).

IMHO, these points combined with the absence of any standardized error handling
for VLA memory allocation failure makes them completely useless for programming
a seroius and robust application. It would be better if alloca() were
standardized.
 
C

Chris Croughton

Speaking of VLAs..

It seems that the standard doesn't specify how VLAs are implemented, although
the most obvious ways are:
1. stack (like the non-portable alloca())
2. malloc() on the heap

A VLA has storage wherever it is used. If you say:

void func(int n)
{
char buff[n];
int x;
...
}

then both x and buff will be allocated in 'automatic' storage (whether
that is on "the stack" depends on whether that's what your compiler
uses).
In both cases you can declare an absurdly large VLA so that there isn't enough
memory to allocate it. I haven't been able to find in the C99 standard what
happens then..

Whatever usually happens when you allocate too much storage. It's
exactly the same as if you declared a fixed length array of the same
size (whatever you system does when it runs out of stack, for instance).
So what is the 'standard' way to handle an out-of-memory condition for VLAs?
There is no place to 'return NULL' as malloc() does..

There isn't one, the same as there isn't a 'standard' way to handle
recursion too deep, or declaring a 2GB 'normal' array in automatic
storage. If you are lucky your program will crash with a system error
and a core dump, if you are unlucky demons will erupt from your computer
and drag you down to the uttermost depths of world slime. The only
difference is that a compiler /might/ be able to trap a silly array size
at compile time with a fixed array (it doesn't have to, but it could be
done).

If you're really lucky the system will say "Oh, you want 57 petabytes of
memory? OK, I'll just go out and buy some for you, wait around a few
years..."

Chris C
 
M

Michael Mair

Zeljko said:
OK, I'm not trying to start a flame here, but why would anyone use VLAs when
they are not guaranteed to always work and you cannot detect that? In all other
aspects of C the programmer is given a chance to handle the error (extending
your line of thought - why check for malloc() returning NULL? the library coud
just terminate the program..)

I think this is a crucial point missing from the standard. The standard
commitee should have taken this into account and specify some kind of
standard behaviour that is under programmer's control (e.g. delivery of a
signal for which a signal handler can be written, at least in the hosted
environments). Otherwise, each implementation will choose its own way of
letting the programmer know about the memory allocation failure (e.g.
#pragma VLA_alloc_failure(handler_func) and similar..) and soon there could
be several incompatible implementations and the C99 code which uses custom VLA
error handling as an extension won't be portable accross them any more..

What's worse, the implementation can *never* guarantee ANY size of VLA because
it depends on the current runtime environment (e.g. in one environment I could
possibly get 16M VLA, and in the other I couldn't get even 64k).

You can't even know in advance the maximum amount of memory needed for VLAs
so it could be preallocated at program startup (or the execution fails if it
can't be preallocated).

IMHO, these points combined with the absence of any standardized error handling
for VLA memory allocation failure makes them completely useless for programming
a seroius and robust application. It would be better if alloca() were
standardized.

In a way, you are right. However, even

double array[7000][7000];

can die at runtime without telling you why and while you still
have enough memory left...
If we added checks and safeguards and error states for each and
everything, then this new C would probably suffer from an even worse
lack of popularity than C99.

Back to VLAs: IMO, they are quite nice for reasonably small array
sizes but for larger array sizes I would always suggest to handle the
memory yourself i.e. explicitly allocate it and maybe implement a sparse
structure.


Cheers
Michael
 
C

Chris Croughton

OK, I'm not trying to start a flame here, but why would anyone use VLAs when
they are not guaranteed to always work and you cannot detect that? In all other
aspects of C the programmer is given a chance to handle the error (extending
your line of thought - why check for malloc() returning NULL? the library coud
just terminate the program..)

No, if you do

int main(int argc, char **argv)
{
long array[10000][10000];
printf("Hello world\n");
return 0;
}

then that is equally likely to fail at runtime with no way of catching
the error (trying to allocate 400 MB or more on the stack will probably
blow it). There is no difference. Or for that matter:

void func(int depth)
{
long array[10000];
if (depth > 0)
func(depth - 1);
}

int main(int argc, char **argv)
{
func(10000);
printf("Hello world\n");
return 0;
}

(it will run out of stack in the recursion).
I think this is a crucial point missing from the standard. The standard
commitee should have taken this into account and specify some kind of
standard behaviour that is under programmer's control (e.g. delivery of a
signal for which a signal handler can be written, at least in the hosted
environments). Otherwise, each implementation will choose its own way of
letting the programmer know about the memory allocation failure (e.g.
#pragma VLA_alloc_failure(handler_func) and similar..) and soon there could
be several incompatible implementations and the C99 code which uses custom VLA
error handling as an extension won't be portable accross them any more..

No, it is very unlikely that any implementors will put in any error
handling for it at all. As I pointed out above, there's nothing to stop
a program from trying to allocate too much on the stack with fixed
length arrays, why should VLAs be any different?

Note that in the version of the language which allows VLAs (C99) it is
also allowed to declare variables after other code:

void func(int len)
{
if (len < 1000)
{
int array[len];
/* do stuff */
}
else
{
/* do failure stuff */
}
}

If you try to force compilers (in the standard) to put in code to catch
every error then the code size will increase and the speed will drop and
you will find that no one uses the language.
What's worse, the implementation can *never* guarantee ANY size of VLA because
it depends on the current runtime environment (e.g. in one environment I could
possibly get 16M VLA, and in the other I couldn't get even 64k).

Yup. The same is true of any automatic variables. I have code which
assumes that a function can allocate a 16kB buffer locally, it will
break on some systems (on the systems where it is intended to run it is
safe, because I know that the stack is big enough there).
You can't even know in advance the maximum amount of memory needed for VLAs
so it could be preallocated at program startup (or the execution fails if it
can't be preallocated).

Nope. You can't know in advance anything at all about the system at run
time, it might crash just trying to load your program. We live in an
uncertain universe...
IMHO, these points combined with the absence of any standardized error handling
for VLA memory allocation failure makes them completely useless for programming
a seroius and robust application. It would be better if alloca() were
standardized.

Except that there are systems on which alloca() is difficult to
implement reliably as well (some programs test at install time whether
alloca() works and use a different mechanism if it doesn't), so the
standard would have to allow alloca() to always fail.

Personally, I think that all of the C99 features are pretty useless
until it can be reasonably guaranteed that almost everywhere will have a
conforming compiler and library. At the moment I can't portably use
VLAs, late declaration of variables, long long or many other features
I'd rather like to use...

Chris C
 
K

Keith Thompson

Michael Mair said:
Zeljko Vrba wrote: [...]
Speaking of VLAs.. It seems that the standard doesn't specify how
VLAs are implemented, although
the most obvious ways are:
1. stack (like the non-portable alloca())
2. malloc() on the heap
In both cases you can declare an absurdly large VLA so that there
isn't enough memory to allocate it. I haven't been able to find in
the C99 standard what happens then.. So what is the 'standard' way
to handle an out-of-memory condition for VLAs? There is no place
to 'return NULL' as malloc() does..

As this is not your problem but the implementation's:
The program will die. If you are lucky, it will do so in a verbose
way. I would not nurse any expectations beyond a size of 64K...

You can't assume that the program will die. Running out of memory for
a VLA invokes undefined behavior, just like running out of memory for
any other automatic variable.

Probably the only way to fix this would be to define a general
exception-handling mechanism for C, and require that running out of
space raises an exception that can be handled by the caller. (You
could provide some specific mechanism for VLAs, but that would be ugly
IMHO, since it wouldn't handle overflows for constant-sized arrays.)
 
Z

Zeljko Vrba

No, it is very unlikely that any implementors will put in any error
handling for it at all. As I pointed out above, there's nothing to stop
a program from trying to allocate too much on the stack with fixed
length arrays, why should VLAs be any different?
Ah, but there's the catch: with fixed-size auto variables only (scalars and
fixed-length arrays, etc.), *theoretically*[1] you could declare that your
program uses at most X bytes of automatic storage.[2] As soon as
variable-length enters the picture, you can make no such guarantees.

[1] I wouldn't go into the correctness of such declaration and how it was
calculated :)

[2] Not an unrealistic assumption. I did write C programs for embedded systems
where I did not use recursion and have always watched not to allocate
more than about 40 bytes of auto storage per function (old versions of
PalmOS have severely restricted stack space).

In another project I had to subordinate the complete program structure to
the underlying processor which had only 8 levels of HW return stack, two
of which I couldn't use because of two external interrupts..

My point is that the programmer can always discipline himself and refrain
from using recursion and allocating too much auto storage. As a consequence,
he knows the maximum function call nesting level and can give a pretty good
_estimate_ of max auto storage required.

So the analogy with "whatever happens when auto storage runs out for too deep
recursion" is not really good because, given enough auto storage and without
VLAs, you *can* guarantee that it will NEVER run out.

Thanks everyone.. it's been a really interesting discussion. When I have first
read about VLAs I immediately noticed the glaring hole: the absence of error
handling. Nice to hear what others have to say about the issue.
 
M

Mark McIntyre

OK, I'm not trying to start a flame here, but why would anyone use VLAs when
they are not guaranteed to always work and you cannot detect that?

TBF, many things are not guaranteed to always work, and many of them don't
give you much chance to detect the error.
In all other aspects of C the programmer is given a chance to handle the error

if only that were true....
 
C

Chris Croughton

No, it is very unlikely that any implementors will put in any error
handling for it at all. As I pointed out above, there's nothing to stop
a program from trying to allocate too much on the stack with fixed
length arrays, why should VLAs be any different?
Ah, but there's the catch: with fixed-size auto variables only (scalars and
fixed-length arrays, etc.), *theoretically*[1] you could declare that your
program uses at most X bytes of automatic storage.[2] As soon as
variable-length enters the picture, you can make no such guarantees.

So don't use it in that particular case. The same as if you want
predictable behaviour with allocating memory in any other way you don't
use malloc (in particular, with malloc and free you can easily end up
with no more than a 'known' amount used but with none available because
of fragmentation).
[1] I wouldn't go into the correctness of such declaration and how it was
calculated :)

[2] Not an unrealistic assumption. I did write C programs for embedded systems
where I did not use recursion and have always watched not to allocate
more than about 40 bytes of auto storage per function (old versions of
PalmOS have severely restricted stack space).

I've also written embedded software, and yes you make sure to not use
features which will blow the stack. We also used fixed pools of memory
so that we didn't get fragmentation, and didn't use malloc/free at all.
In another project I had to subordinate the complete program structure to
the underlying processor which had only 8 levels of HW return stack, two
of which I couldn't use because of two external interrupts..
Yup.

My point is that the programmer can always discipline himself and refrain
from using recursion and allocating too much auto storage. As a consequence,
he knows the maximum function call nesting level and can give a pretty good
_estimate_ of max auto storage required.

Exactly. So if the programmer is in a situation where over-allocating
stack is a problem, they don't use VLAs. Not a problem, nothing in the
standard says that you have to use them, they are there for people who
do want to use them and can do so responsibly.
So the analogy with "whatever happens when auto storage runs out for too deep
recursion" is not really good because, given enough auto storage and without
VLAs, you *can* guarantee that it will NEVER run out.

You can guarantee it even with VLAs as well, just as much as you can
using alloca (on systems where that is available). But you asked about
what happens when it is too big, and the answer /is/ "the same as always
happens when the stack isn't big enough". Whatever that is.

Personally, I doubt that I'll ever /need/ to use VLAs. I might use them
in a few cases where I know that I need a local buffer and know that it
wil be a reasonable size, but I might as well use fixed length arrays or
malloc for that. I've never had a need to use alloca() either...
Thanks everyone.. it's been a really interesting discussion. When I have first
read about VLAs I immediately noticed the glaring hole: the absence of error
handling. Nice to hear what others have to say about the issue.

It's a general 'hole' to do with automatic variables. I'd rather like
an exception to be thrown (if C had exceptions!) whenever the stack (or
equivalent) gets exhausted, just to have something to catch, but
mandating it would cause too much grief. Having a general error
mechanism would be a good thing...

Chris C
 
K

Kenny McCormack

OK, I'm not trying to start a flame here, but why would anyone use VLAs
when they are not guaranteed to always work and you cannot detect that?

TBF, many things are not guaranteed to always work, and many of them don't
give you much chance to detect the error.
In all other aspects of C the programmer is given a chance to handle the
error
[/QUOTE]

I just have 2 comments:
1) Yes, the problem with VLAs is exactly the same as the general
problem of automatic variables. Namely, that you can't ever know how much
is too much.
2) There *are* some places in the standard where they say things
like: A conforming implementation must support at least X (with respect to
feature Y). It seems like there could be such a requirement in terms of
how much automatic variable space you can assume.
 
J

Jack Klein

Speaking of VLAs..

It seems that the standard doesn't specify how VLAs are implemented, although
the most obvious ways are:
1. stack (like the non-portable alloca())
2. malloc() on the heap

In both cases you can declare an absurdly large VLA so that there isn't enough
memory to allocate it. I haven't been able to find in the C99 standard what
happens then..

So what is the 'standard' way to handle an out-of-memory condition for VLAs?
There is no place to 'return NULL' as malloc() does..

Others have already pointed out that you face the same issue with
automatic arrays with fixed size. Possible consequences of the
undefined behavior are some sort of trap and orderly shut down by the
operating system on platforms with memory protection, to just plain
overwriting something else, such as other data or code areas, with who
knows what results.

I did not read all the replies, but I have not noticed anyone pointing
out that there are platforms out there, Linux among them, that do lazy
allocation. They may return a non-null pointer from malloc() even
though they do not really give the memory to the application. Later,
when the application actually tries to access the memory, the system
starts killing it or other, unrelated programs.

I do not know, because I am not a member of the C standard committee,
but I think it likely that VLAs are there at the insistence of those
who claim that they cannot abide the lack of alloca(), or equivalent.
Their programs cannot possible survive the overhead of actually
calling malloc().

Say for the moment that there is some real justification here. It
seems to me that this position only makes sense in a program that
performs a large number of small allocations that are only needed for
relative short lengths of time.

If you are going to allocate an array of 1,000,000 doubles, most
likely just initializing the data in the array to meaningful values
far outweighs the allocation, even if malloc() makes an underlying
system call. If such an array is around for much of the life of a
program, it is actually far more efficient to allocate it statically,
which gives the bonus of default initializing all the elements.

So it seems to me that if one wanted to use VLAs at all, the most
prudent strategy would be to use them for relatively small blocks of
memory with relatively short life times.

Note also, that alloca(), which VLAs supercede in a standard way, have
no more guarantee than VLAs do.

Finally, this could well become a QOI issue. If enough people are
convinced that they must replace all other dynamic allocations with
VLAs with some vague, unproven assertions about efficiency, and they
put enough pressure on compiler vendors, most likely the vendors will
respond.

Here's what an implementation could do when it encountered the
definition of a VLA. It could call some sort of platform-specific
run-time library function that would somehow determine whether or not
the space is available for the VLA. If not, it could do something
instead, such as make a system call to increase the space available,
or call malloc() in the normal fashion. Of course this adds at least
some of the overhead that the "efficiency" proponents say that they
can't stand. So maybe the VLA advocates wouldn't like that at all.

The C standard does not recommend against using VLAs at all, or for
objects above a certain size. It also doesn't recommend gets(), or
does it recommend against using scanf("%s", char_pointer).

This is C, after all, and one of the first principles is that you
don't pay for what you don't use. If you want full protection for
dynamic allocation (except on systems with lazy allocation), then you
call malloc() or siblings and pay for it. If you don't want to pay
that price, you don't have to, you can use VLAs and take your chances.

As Heinlein was fond of pointing out, "TANSTAAFL".
 
W

William Ahern

Jack Klein said:
I did not read all the replies, but I have not noticed anyone pointing
out that there are platforms out there, Linux among them, that do lazy
allocation. They may return a non-null pointer from malloc() even
though they do not really give the memory to the application. Later,
when the application actually tries to access the memory, the system
starts killing it or other, unrelated programs.
Here's what an implementation could do when it encountered the
definition of a VLA. It could call some sort of platform-specific
run-time library function that would somehow determine whether or not
the space is available for the VLA. If not, it could do something
instead, such as make a system call to increase the space available,
or call malloc() in the normal fashion. Of course this adds at least
some of the overhead that the "efficiency" proponents say that they
can't stand. So maybe the VLA advocates wouldn't like that at all.

In a similar vein (or maybe not, since one is evil and the other is a
nice-to-have), Linux can grow the stack dynamically. So, it effectively does
the same as above.

Speaking of the evil OOM killer in Linux, I got a smile out of reading this
on Kernel Traffic:

Edjard Souza Mota also pointed out that this patch only moved the
OOM killer's ranking algorithm to user-space, not the killer itself.
He said, "In that sense the approach is different and might be worth
testing, mainly for cases where we want to allow better policies of
ranking. For example, an embedded device with few resources and
important different running applications: whic one is the best? To
my understanding the current ranking policy does not necessarily
chooses the best one to be killed." This made more sense to Marcelo,
and he felt the patch was more worth considering in this case. Folks
continued discussing the technical details for awhile, with no real
consensus on what the best solution would be.

The day when they find the answer to which program to kill will be an
interesting day indeed. The notion that any embedded device using Linux
would keep lazy allocation enabled baffles me; I would hope that none do.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top