'Dynamic' function calls

[BartC]

I have a requirement to call a function where:

o The function address is not known until runtime
o Neither are the number and type of the parameters
o Neither is the type of the function result
o (And neither is the calling convention, but it will likely be one of two)

So nothing too difficult... But I want to avoid sprawling code that tries to
enumerate all the possibilities.

(This to help an interpreted language call a foreign function in a
run-time-loaded dynamic library. It only needs to work on two platforms both
of which have a natural 32-bit int size)

The input parameters are easily dealt with, by letting all possibilities be
represented by an array of 0 to N 32-bit ints. N can be at least 12. But I
can't see any way in C of calling a function pointer with a variable number
of parameters, without writing at least N+1 different calls.

The function return type can be similarly limited to 4 possibilities: none,
32-bit, 64-bit, and 64-bit floating point (I think the latter uses a
different return register on the two platforms I have in mind). While the
call types will be C-language (I assume the library will use the same
convention as my C program), and whatever the OS might use.

So I can probably do this with only 13 x 4 x 2 lots of calls (about 100).
But if there's any way of simplifying this, without dropping into assembly
code, that would be appreciated!

 

[Anders Wegge Keller]

The input parameters are easily dealt with, by letting all
possibilities be represented by an array of 0 to N 32-bit ints. N
can be at least 12. But I can't see any way in C of calling a
function pointer with a variable number of parameters, without
writing at least N+1 different calls.

What happens if you call a two-parameter function with (a1, a2, a3,
a4, a5, ...., aN) ?

We are way out of defined territory here, but most ABI's I know of,
would simply ignore the extra parameters. So you might be lucky that
your supported platforms' ABI would accept such an call.

 

[BartC]

What happens if you call a two-parameter function with (a1, a2, a3,
a4, a5, ...., aN) ?

We are way out of defined territory here, but most ABI's I know of,
would simply ignore the extra parameters. So you might be lucky that
your supported platforms' ABI would accept such an call.

I was wondering that, but didn't dare ask! On my main machine (Windows)
params are pushed right-to-left which helps. The other (Linux with gcc) I'll
have to check out, although I haven't even sorted the mysteries of dlopen()
yet..

Will need to do some tests I think. (I don't need to push 12 params every
time; either 4, 8 or 12 for example will reduce the code considerably).

 

[Ian Collins]

I have a requirement to call a function where:

o The function address is not known until runtime
o Neither are the number and type of the parameters
o Neither is the type of the function result
o (And neither is the calling convention, but it will likely be one of two)

So nothing too difficult... But I want to avoid sprawling code that tries to
enumerate all the possibilities.

This isn't that unusual. The UNIX dlsym() returns all objects,
including function addresses as void*.

(This to help an interpreted language call a foreign function in a
run-time-loaded dynamic library. It only needs to work on two platforms both
of which have a natural 32-bit int size)

The input parameters are easily dealt with, by letting all possibilities be
represented by an array of 0 to N 32-bit ints. N can be at least 12. But I
can't see any way in C of calling a function pointer with a variable number
of parameters, without writing at least N+1 different calls.

I have a library that uses dlsym() to find named functions in an XML-RPC
call. In my case, I do build an arrays of 0,1,2 and 3 parameter callers.

The function return type can be similarly limited to 4 possibilities: none,
32-bit, 64-bit, and 64-bit floating point (I think the latter uses a
different return register on the two platforms I have in mind). While the
call types will be C-language (I assume the library will use the same
convention as my C program), and whatever the OS might use.

In my case, I use seven return types used by XML-RPC, which does result
in quite a large matrix. The code (using C++ templates) is quite
compact, but takes a while to compile! This isn't usually a problem as
it just keeps one core busy while the rest of the library compiles on
the rest..

 

[Rui Maciel]

I have a requirement to call a function where:

o The function address is not known until runtime
o Neither are the number and type of the parameters
o Neither is the type of the function result
o (And neither is the calling convention, but it will likely be one of
two)

So nothing too difficult... But I want to avoid sprawling code that tries
to enumerate all the possibilities.

(This to help an interpreted language call a foreign function in a
run-time-loaded dynamic library. It only needs to work on two platforms
both of which have a natural 32-bit int size)

The input parameters are easily dealt with, by letting all possibilities
be represented by an array of 0 to N 32-bit ints. N can be at least 12.
But I can't see any way in C of calling a function pointer with a variable
number of parameters, without writing at least N+1 different calls.

The function return type can be similarly limited to 4 possibilities:
none, 32-bit, 64-bit, and 64-bit floating point (I think the latter uses a
different return register on the two platforms I have in mind). While the
call types will be C-language (I assume the library will use the same
convention as my C program), and whatever the OS might use.

So I can probably do this with only 13 x 4 x 2 lots of calls (about 100).
But if there's any way of simplifying this, without dropping into assembly
code, that would be appreciated!

Is it possible to wrap that function? If that is possible then you could
solve the parameters issue by using a struct that stores a list of
parameters to pass an indeterminate number of parameters, and for the return
type you could use void pointer passed as a parameter and then let the
return value of the wrapper point which data type was passed, to be able to
cast that void pointer to something meaningful.

For example:

<pseudo-ish code>
struct Params {
int n;
int *parameters;
};

enum WrapperReturnType {
WRT_VOID,
WRT_FP32,
WRT_FP64
};

enum WrapperReturnType
wrapper(struct Params *parameters, void *returned_value);
</pseudo-ish code>


Rui Maciel

 

[Shao Miller]

I have a requirement to call a function where:

o The function address is not known until runtime
o Neither are the number and type of the parameters
o Neither is the type of the function result
o (And neither is the calling convention, but it will likely be one of two)

So nothing too difficult... But I want to avoid sprawling code that
tries to
enumerate all the possibilities.

(This to help an interpreted language call a foreign function in a
run-time-loaded dynamic library. It only needs to work on two platforms
both
of which have a natural 32-bit int size)

The input parameters are easily dealt with, by letting all possibilities be
represented by an array of 0 to N 32-bit ints. N can be at least 12. But I
can't see any way in C of calling a function pointer with a variable number
of parameters, without writing at least N+1 different calls.

The function return type can be similarly limited to 4 possibilities: none,
32-bit, 64-bit, and 64-bit floating point (I think the latter uses a
different return register on the two platforms I have in mind). While the
call types will be C-language (I assume the library will use the same
convention as my C program), and whatever the OS might use.

So I can probably do this with only 13 x 4 x 2 lots of calls (about 100).
But if there's any way of simplifying this, without dropping into assembly
code, that would be appreciated!

It doesn't seem that portability is a concern, so perhaps you can
investigate the possibility of building an automatic VLA inside some
curly braces and then passing _no_ arguments, then see if the called
function picks up its arguments from the VLA in the caller. If it
works, it might reduce the combinations to 4 x 2.

(A VLA could even be one of 'unsigned char' and populated with the
object representations of "the parameters" the called function is
expecting, that way, all "params" needn't have the same size.)

I haven't used this strategy before on any platform, so please consider
that when considering the potential for wasting time.

- Shao Miller

 

[glen herrmannsfeldt]

What happens if you call a two-parameter function with (a1, a2, a3,
a4, a5, ...., aN) ?

We are way out of defined territory here, but most ABI's I know of,
would simply ignore the extra parameters. So you might be lucky that
your supported platforms' ABI would accept such an call.

For K&R C, that pretty much has to be true for varargs to work.

With ANSI C, at least C89, it only has to be true for actual
varargs routines. A completely different calling convention
could be used for non-varargs functions. But on most systems
the same convention is used.

The 8086 (and successors) have a RET (return) instruction that
can also pop a specified number of (16 bit) words off the stack.

The early 8086 compilers, for Fortran and Pascal, used this,
as they don't have any variable argument calls. That is, the
callee pops the arguments off the stack.

In the transition to 32 bit code, this evolved to the stdcall
( http://en.wikipedia.org/wiki/Stdcall#stdcall ) convention.

When C compilers started to appear for 8086, with the requirement
to support varargs (and before ANSI C), a new calling convention
was defined such that the caller (which knows the number of arguments)
pops them off the stack.

Along with the evolution (and naming) of stdcall, this evolved
to be named cdecl, where the caller pops the arguments.

Some compilers allow one to specify the calling convention with
the function prototype (or definition), such that they can be
mixed within a program.

For C functions that aren't varargs, it would be possible to use
the stdcall convention, though I don't know of any compilers that do.

With the cdecl convention, you should be safe passing extra arguments
that are ignored by the callee. The standard doesn't say anything
about this, though.

-- glen

 

[BartC]

When C compilers started to appear for 8086, with the requirement
to support varargs (and before ANSI C), a new calling convention
was defined such that the caller (which knows the number of arguments)
pops them off the stack.

Along with the evolution (and naming) of stdcall, this evolved
to be named cdecl, where the caller pops the arguments.

Some compilers allow one to specify the calling convention with
the function prototype (or definition), such that they can be
mixed within a program.

For C functions that aren't varargs, it would be possible to use
the stdcall convention, though I don't know of any compilers that do.

With the cdecl convention, you should be safe passing extra arguments
that are ignored by the callee. The standard doesn't say anything
about this, though.

That's a good point. On my machine, calls to normal C functions have the
stack adjusted by the caller. So the extra params are popped.

But with C calls to Win32 functions, for example, this is handled by the
callee, which will only pop what are expected. (And a test showed this
doesn't work. In any case, there is a further problem that the names of such
functions, when passed to the linker, have the number of parameter bytes
encoded into them! Circumventing that just caused a crash.)

 

[Shao Miller]

For C functions that aren't varargs, it would be possible to use
the stdcall convention, though I don't know of any compilers that do.

stdcall is used pretty frequently in Microsoft-land. - Shao

 

[glen herrmannsfeldt]

On 12/21/2012 19:20, glen herrmannsfeldt wrote:

stdcall is used pretty frequently in Microsoft-land. - Shao

Yes, but are there C compilers that use stdcall without any special
attributes or compiler options requesting it?

Even more, that use it for libc?

I know it is common for Windows system calls, but then since it is
closed source we don't know that it is C.

-- glen

 

[Malcolm McLean]

I have a requirement to call a function where:

o The function address is not known until runtime

o Neither are the number and type of the parameters

o Neither is the type of the function result

o (And neither is the calling convention, but it will likely be one of two)

It can't be done. It's one of the few things which is possible, in fact quite
easy, in assembler, and C provides no hook into. You simply cannot build
a call list at run time.
What you need to do is to write a little assembler routine, with the C
signature
void call(void (*fptr)(void), char *signature, void *result, ...)

You probably want printf() style signatures, so to call a function taking an
integer, a double and a structure pointer

call(myfunction, "%f%d%p", 0, 0.5, 42, &mystruct);

How do you write the function call? You've got to know the C calling
convention. Typically this says "put the first few arguments in registers
and the rest on the stack". So you have to know what C regards as the stack,
and which registers take which arguments. Normally the scheme is quite simple,
real arguments go in float registers, integers and pointers in general
purpose registers.

 

[Nobody]

I have a requirement to call a function where:

o The function address is not known until runtime
o Neither are the number and type of the parameters
o Neither is the type of the function result
o (And neither is the calling convention, but it will likely be one of two)

You might want to look at libffi rather than re-inventing the wheel.

A number of high-level languages use it for their foreign function
interface (e.g. Python's ctypes module).

 

[Kenny McCormack]

You might want to look at libffi rather than re-inventing the wheel.

A number of high-level languages use it for their foreign function
interface (e.g. Python's ctypes module).

Yes, as others have verbosely told you, you can't do this in "Standard C".

Luckily, that is not really much of a concern, once you get into this sort
of thing.

Anyway, I've never heard of "libffi" (as posted by "Nobody"), but I've used
"ffcall" extensively - specifcally the "avcall" mechanism. I've compiled
this on lots of different architectures and it has worked fine [*].
Strongly recommended.

Google for "ffcall-1.10.tgz" - which is, I think, the latest (last) version.

[*] FWIW, it didn't compile on one specific ARM platform that I've tried
recently. I'm hoping to hear back from the author about the status of this.

--
Religion is regarded by the common people as true,
by the wise as foolish,
and by the rulers as useful.

(Seneca the Younger, 65 AD)

 

[BartC]

It can't be done. It's one of the few things which is possible, in fact
quite
easy, in assembler, and C provides no hook into. You simply cannot build
a call list at run time.

It looks like it. But I'd prefer to keep the program 'pure', and not use
assembler for now. (I develop on a Windows PC, but I recently tested this
~12Kloc C program on a Linux/ARM development board, and it ran perfectly**
first time! Also I don't know ARM assembly well enough...)

How do you write the function call? You've got to know the C calling
convention. Typically this says "put the first few arguments in registers
and the rest on the stack". So you have to know what C regards as the
stack,
and which registers take which arguments. Normally the scheme is quite
simple,
real arguments go in float registers, integers and pointers in general
purpose registers.

(With 32-bit systems, I found float arguments just go on the stack like
everything else. But a float result might be in a floating point register.)

What I think I'll try is to reduce the number of return types to just two:
64-bit int, and 64-bit float. And hope the 64-bit return will be meaningful
when the return type is void, or 32-bits (just ignore the top 64 or 32-bits
of the return value!).

(**Apart from being too slow. That was due to alignment problems, soon
fixed.)

 

[BartC]

You might want to look at libffi rather than re-inventing the wheel.

(I think my entire project can be considered reinventing the wheel! (And,
actually I only recently switched to using C for it instead of my own
languages.)

But, my wheels tend to be smaller, simpler, faster and (imo, and for my
purposes) better!)

A number of high-level languages use it for their foreign function
interface (e.g. Python's ctypes module).

I've had a quick look (at ffcall, which is where google led me), this does
seem to be depend on using 'asm'. But it's also quite a complex package,
which may or may not work under Windows, and I'll have to weigh up whether
trying to make use of it is better than just writing a few dozen lines of
repetitive C, or even a few dozen lines of my own ASM.

 

[Kenny McCormack]

I've had a quick look (at ffcall, which is where google led me), this does
seem to be depend on using 'asm'. But it's also quite a complex package,
which may or may not work under Windows, and I'll have to weigh up whether
trying to make use of it is better than just writing a few dozen lines of
repetitive C, or even a few dozen lines of my own ASM.[/QUOTE]

ffcall (actually, specicially, avcall) works under Windows. I compiled it
for Windows long ago, using MSVC. Builds out of the box.

It has worked on every platform I've tried it on right up until I tried it
on one of my ARM boards a few days ago. Still not sure about the resolution
of that issue yet...

--
Here's a simple test for Fox viewers:

1) Sit back, close your eyes, and think (Yes, I know that's hard for you).
2) Think about and imagine all of your ridiculous fantasies about Barack Obama.
3) Now, imagine that he is white. Cogitate on how absurd your fantasies
seem now.

See? That wasn't hard, was it?

 

[Malcolm McLean]

If the list of possible combinations is not too big, just write down
every possible combination, and select the appropriate one to execute.



He indicated that he knew about this solution, and was trying to avoid
it, because the list of possible combinations is fairly large - but you
didn't quote that part of his message. The part you did quote describes
a soluble problem (though not an easy one).

Theoretically you're right. The assembler version involves writing every
possible combination. But, if the calling convention genuinely was O(M^N)
complexity, where M was the number of types and N the number of arguments,
then a compiler would be very difficult to write.
In fact the convention will be something like "all integer and address
arguments go in r1, r2, r3, r4, then on the stack, all real arguments in
f1, f2, f3, f4 then on the stack. So it's quite easy to build a call list,
using these rules, and you can as easily pass two arguments as twenty.
Doing it from C is the O(M^N) solution.

 

[JimB]

I have a requirement to call a function where:

You are lying, to yourself. You have no such requirement. I am not saying
that you are trying to maintain semblence of existence, but I'll leave that
up to "your country". "You do have a Country, boy, don't you? Well, if you
don't, we'll fix you right up with one pronto... now bend over and spread.".

 

[glen herrmannsfeldt]

(snip on calling functions with an unknown, at compile time, number
of arguments, possibly of unknown type.)

It looks like it. But I'd prefer to keep the program 'pure', and not use
assembler for now. (I develop on a Windows PC, but I recently tested this
~12Kloc C program on a Linux/ARM development board, and it ran perfectly**
first time! Also I don't know ARM assembly well enough...)

There are too many different ways to do it, so you can't really be sure.

For some time, at least, all the arguments were passed in the same way,
but now it is more common for some to be passed in registers. That does
complicate things.

(With 32-bit systems, I found float arguments just go on the stack like
everything else. But a float result might be in a floating point register.)

Even worse, if that register is on the x87 stack. You can't just ignore
it, as sometime later, likely at an unexpected time and place, a
floating point stack overflow will be detected.

I do remember random number generators for OS/360 Fortran that would
return either a floating point value between 0 and 1, or a 32 bit
integer, depending on declaring the function INTEGER or REAL.

The (assembler) function returned values in both general and floaing
point registers, and the compiler used the appropriate one.

Don't try that on IA32.

What I think I'll try is to reduce the number of return types to just two:
64-bit int, and 64-bit float. And hope the 64-bit return will be meaningful
when the return type is void, or 32-bits (just ignore the top 64 or 32-bits
of the return value!).

Many 32 bit compilers will pass an additional argument to store the
result in if the result type is more than 32 bits. (Such as a struct.)
That might not be true for 64 bit compilers.

(**Apart from being too slow. That was due to alignment problems, soon
fixed.)

-- glen

 

[glen herrmannsfeldt]

(snip regarding different function return types.)

If the list of possible combinations is not too big, just write down
every possible combination, and select the appropriate one to execute.

Reminds me of the PL/I ENTRY statement for functions. If different ENTRY
points have different return types, the compiler will generate code to
convert each posible type returned (argument to a RETURN statement) to
each ENTRY type. (I suppose a compiler could do enough flow analysis
to figure out which conversions could be needed, but often that
can't be done.)

He indicated that he knew about this solution, and was trying to avoid
it, because the list of possible combinations is fairly large - but you
didn't quote that part of his message. The part you did quote describes
a soluble problem (though not an easy one).

Theoretically, the number of possible return types is infinite, but
in practical cases not quite. (Mostly due to an unknown number of
levels of indirection, which can easily be handled as (void *)).

-- glen