gamedevlists-general

[GD-General] variadic functions

[Brett B. <res...@ga...>]

Hiya,
I'm trying to bind our scripting engine to our game engine and having
trouble with variadic functions; namely I can't figure out how to call a
C-implemented variadic function from our scripting engine where the
parameters are not known at compile time of the engine.  Of course
ideally I could implement this in C itself but it seems I need to
implement some assembly code to push the parameters correctly, and so
maybe I shouldn't worry about the whole implementation, but I would like
to understand how this works.

For purposes of discussion, I'll assume the implementation is for a GCC
compatible implementation.  Basically, variadic functions are
implemented in two steps.

1.  The caller (calling the variadic function) simply pushes the
parameters and calls the variadic function.

2.  The callee (the variadic function) uses the following four basic
functions to be able to use variable arguments (GCC compatibility
style):

void va_start(va_list ap, last);
type va_arg(va_list ap, type);
void va_end(va_list ap);
void va_copy(va_list dest, va_list src);

The compiler strips the "..." off the back and makes a normal function
out of the variadic one, and the preceeding four functions are then used
to access these unknown parameters.  The start function takes the last
known parameter as a value so at first glance it seems the function
knows how many paramters there are because because it can compare the
last known argument with the value on the stack.  The problem is that
since the parameters were pushed in reverse order how would it know how
many parameters there are?  If it were in direct order, you could take
the stack pointer minus the stack pointer at the last parameter position
(which is provided) and the diffference would be the number of
arguments.  But since they are reversed how does the va_arg function
know when to stop?

I wrote a test sample for the PlayStation 2, GameCube and PC (Intel) and
it seems that the caller does nothing more than push the values onto the
stack as a normal function would, making sure not to use registers.

Has anybody else run into this before?  How did you solve it?

Thanks,
Brett

Re: [GD-General] variadic functions

[Andras B. <bn...@ma...>]

Hi Brett,

> The compiler strips the "..." off the back and makes a normal function
> out of the variadic one, and the preceeding four functions are then used
> to access these unknown parameters.  The start function takes the last
> known parameter as a value so at first glance it seems the function
> knows how many paramters there are because because it can compare the
> last known argument with the value on the stack.  The problem is that
> since the parameters were pushed in reverse order how would it know how
> many parameters there are?  If it were in direct order, you could take
> the stack pointer minus the stack pointer at the last parameter position
> (which is provided) and the diffference would be the number of
> arguments.  But since they are reversed how does the va_arg function
> know when to stop?

The answer is simple: It does not know! :(

That's exactly why vararg functions always use the cdecl calling
convention. This requires that the caller cleans the stack, because
the caller always knows the number of params on the stack, while the
calle doesn't (in the case of vararg).

Think of printf! The first param is supposed to be a pointer to a
format string. Reading that string printf determines how many (and of
what type) values it has to read from the stack. I'm pretty sure that
every one of us crashes a program every once in a while because of
unmatching params and format string. The reason for this is that there
is just no way printf can verify the stuff you gave is valid.

> Has anybody else run into this before?  How did you solve it?

Well, you could use a format string like printf, or something similar.

If you do this for scripting, then your script binding engine knows
all the params, so it might be a good idea to generate the format
description string automatically. This way the script writers could
call the function with any number of params without explicitly
describing their type and number in a format string.

The ultimate solution would be of course the introduction of a new
calling convention, but I don't really know who's responsibility is
this... anyone? :)


Andras

Re: [GD-General] variadic functions

[Brett B. <res...@ga...>]

Andras,
Thanks for the post.  Pretty much my conclusion too.  But I'm not using
this for printf so I need a generalized solution, although I could
theoretically create bindings for every possible case (which is normal
for scripting engine binding anyway).  Since the callee doesn't know how
many parameters there are, it seems I just need to push the values to
the stack in assembly code and then call the variadic function.  Does
that sound right?
Brett

----- Original Message ----- 
From: "Andras Balogh" <bn...@ma...>
To: <gam...@li...>
Sent: Thursday, November 27, 2003 4:28 PM
Subject: Re: [GD-General] variadic functions


> Hi Brett,
>
> > The compiler strips the "..." off the back and makes a normal
function
> > out of the variadic one, and the preceeding four functions are then
used
> > to access these unknown parameters.  The start function takes the
last
> > known parameter as a value so at first glance it seems the function
> > knows how many paramters there are because because it can compare
the
> > last known argument with the value on the stack.  The problem is
that
> > since the parameters were pushed in reverse order how would it know
how
> > many parameters there are?  If it were in direct order, you could
take
> > the stack pointer minus the stack pointer at the last parameter
position
> > (which is provided) and the diffference would be the number of
> > arguments.  But since they are reversed how does the va_arg function
> > know when to stop?
>
> The answer is simple: It does not know! :(
>
> That's exactly why vararg functions always use the cdecl calling
> convention. This requires that the caller cleans the stack, because
> the caller always knows the number of params on the stack, while the
> calle doesn't (in the case of vararg).
>
> Think of printf! The first param is supposed to be a pointer to a
> format string. Reading that string printf determines how many (and of
> what type) values it has to read from the stack. I'm pretty sure that
> every one of us crashes a program every once in a while because of
> unmatching params and format string. The reason for this is that there
> is just no way printf can verify the stuff you gave is valid.
>
> > Has anybody else run into this before?  How did you solve it?
>
> Well, you could use a format string like printf, or something similar.
>
> If you do this for scripting, then your script binding engine knows
> all the params, so it might be a good idea to generate the format
> description string automatically. This way the script writers could
> call the function with any number of params without explicitly
> describing their type and number in a format string.
>
> The ultimate solution would be of course the introduction of a new
> calling convention, but I don't really know who's responsibility is
> this... anyone? :)
>
>
> Andras
>
>
>
> -------------------------------------------------------
> This SF.net email is sponsored by: SF.net Giveback Program.
> Does SourceForge.net help you be more productive?  Does it
> help you create better code?  SHARE THE LOVE, and help us help
> YOU!  Click Here: http://sourceforge.net/donate/
> _______________________________________________
> Gamedevlists-general mailing list
> Gam...@li...
> https://lists.sourceforge.net/lists/listinfo/gamedevlists-general
> Archives:
> http://sourceforge.net/mailarchive/forum.php?forum_id=557

Re: [GD-General] variadic functions

[Andras B. <bn...@ma...>]

Brett,

What do you mean by using a generalized solution? In the case of a
vararg function, you have to describe the parameters somehow! And the
easiest way to do it is via a string: for example "iicfds" would mean:
(int, int, char, float, double, char*).

Of course this means that you have to prepare your function to be able
to interpret this format string, and act appropriately, but I think
this should not be a problem as there is no point in writing a vararg
function that can't handle different sets of parameters...

In case you only have a couple of variants, then it might be more
efficient to make different versions of the function to handle each
case, and bind them separately.

And to answer your last question: Yes, simply pushing the values and
calling the func should do the job. But don't forget to pop the values
from the stack after the func returned, as it's your (the caller's)
job to clean-up the stack!!!

Cheers,


Andras

Thursday, November 27, 2003, 11:02:46 AM, Brett Bibby wrote:

> Andras,
> Thanks for the post.  Pretty much my conclusion too.  But I'm not using
> this for printf so I need a generalized solution, although I could
> theoretically create bindings for every possible case (which is normal
> for scripting engine binding anyway).  Since the callee doesn't know how
> many parameters there are, it seems I just need to push the values to
> the stack in assembly code and then call the variadic function.  Does
> that sound right?
> Brett

Re: [GD-General] variadic functions

[Thatcher U. <tu...@tu...>]

On Nov 27, 2003 at 10:22 +0800, Brett Bibby wrote:
> 
> I'm trying to bind our scripting engine to our game engine and having
> trouble with variadic functions

I'm surprised that you have variadic functions on the C/C++ side of
your engine.  What are they used for?

Anyway, one possible approach is to do the wrapping in reverse.
I.e. rewrite those functions avoiding the C/C++ implementation of
variadic parameters, and instead use your scripting language's
parameter-passing mechanism.  Bind the functions directly to your
scripting language.  Provide wrappers for calling them from C/C++.

E.g. let's say you're using Lua, and you have an add() function that
takes an arbitrary number of doubles and returns their sum:

double add(int arg_count, ...) { /* ... */ }

You can implement that using the va_ macros, but instead of doing
that, you could implement it using the Lua API:

int add_lua(lua_State* L)
{
	int arg_count = (int) luaL_checknumber(1);
	double sum = 0;
	for (int i = 0; i < arg_count; i++)
	{
		sum += luaL_checknumber(2 + i);
	}
	lua_pushnumber(L, sum);
	return 1;
}

// Wrapper for calling from C++.
double add(int arg_count, ...)
{
	// Seat of the pants code, probably incorrect!
	lua_pushnumber(L, arg_count);
	va_list argp;
	va_start(argp, arg_count);
	for (int i = 0; i < arg_count; i++)
	{
		lua_pushnumber(L, va_arg(argp, double));
	}
	va_end(argp)
	add_lua(L);
	double result = luaL_tonumber(L, -1);
	lua_pop(arg_count + 2);
	return result;
}
// etc.

-- 
Thatcher Ulrich
http://tulrich.com