Temp opaque variable trick

[brncrs]

Hi list!
I'm writing out a program where string's are handled by opaque
structure and each string variable is allocated/deallocated
dynamically
using functions like `string_new' and `string_delete'.

In some part of this program I need variables of this type
only to let an argument to a function, and alloc/dealloc this variable
manually don't make me very happy.

Example:

typedef struct string* _Opaque_string_type;
typedef struct device* _Opaque_device_type;

...

extern _Opaque_string_type* string_new(const char *, ...);
extern void string_delete(_Opaque_string_type *);

extern _Device_type* device_open(_Opaque_string_type* p_path);

...

void foo(void)
{
_Opaque_string_type str;
_Opaque_device_type dev;

str = string_new("/dev/ttyACM0");
dev = device_open( str );
string_delete(str);
}

The best solution I got is to let to the `string_new' function the
address of a buffer where to store the variable:

Example:

typedef struct string* _Opaque_string_type;
typedef struct device* _Opaque_device_type;

...

extern _Opaque_string_type* string_new(void *, size_t, const
char *, ...);

extern _Device_type* device_open(_Opaque_string_type* p_path);

...

extern _Device_type* device_open(_Opaque_string_type* p_path);

void foo(void)
{
char buf[64];
_DeviceType *dev = device_open( string_new(buf, sizeof
(buf), "/dev/ttyACM0") );
}

So the string is stored in `buf' and I don't need to free them with
`string_delete'. Internally the string_new function organize memory in
this way:

+--------+----------------------
+-------------------------------------+
| size | data_ptr | "/dev/
ttyACM0" |
+--------+-----------+----------
+-------------------------------------+
|____________^

And cast initial address to _Opaque_string_type*.

Unfortunately, with this approach the string length is limited by the
size of the buffer, and I must declare buffer explicit.

I read about function `alloca', but I can't use it in a function
parameter list.

I will improve this method, someone can suggest a trick to me?


Excuse me for horrible English language!

Thanks in advance!


Christian.

 

[brncrs]

Hi list!
        I'm writing out a program where string's are handled by opaque
structure and each string variable is allocated/deallocated
dynamically
using functions like `string_new' and `string_delete'.

In some part of this program I need variables of this type
only to let an argument to a function, and alloc/dealloc this variable
manually don't make me very happy.

        typedef struct string* _Opaque_string_type;
        typedef struct device* _Opaque_device_type;

     Poor choice of names: These are both "reserved for any use,"
meaning that the C implementation might already be using them for
its own purposes.  It's not quite as bad as using `printf' and
`sqrt' as names for your struct pointer types, but nearly.

        ...

        extern _Opaque_string_type* string_new(const char *, ....);
        extern void string_delete(_Opaque_string_type *);

        extern _Device_type* device_open(_Opaque_string_type* p_path);

        ...

        void foo(void)
        {
                _Opaque_string_type str;
                _Opaque_device_type dev;

                str = string_new("/dev/ttyACM0");

     This can't be right.  `str' is an `_Opaque_string_type', which
turns out to be a `struct string*'.  The string_new() function returns
an `_Opaque_string_type*', which is a `struct string**'.  Those are two
different types of pointers, and there's no implicit conversion between
them.  What this means to me is that the "example" is faulty, and I
don't know what it's supposed to be illustrating.

                dev = device_open( str );

     Also can't be right, for the same reason: Argument type does
not match (and can't be silently converted to) parameter type.

                string_delete(str);

     Also can't be right; same reason.

        }

The best solution I got is to let to the `string_new' function the
address of a buffer where to store the variable:

        typedef struct string* _Opaque_string_type;
        typedef struct device* _Opaque_device_type;

        ...

        extern _Opaque_string_type* string_new(void *, size_t, const
char *, ...);

        extern _Device_type* device_open(_Opaque_string_type* p_path);

        ...

        extern _Device_type* device_open(_Opaque_string_type* p_path);

        void foo(void)
        {
                char buf[64];
                _DeviceType *dev = device_open( string_new(buf, sizeof
(buf), "/dev/ttyACM0") );
        }

So the string is stored in `buf' and I don't need to free them with
`string_delete'. Internally the string_new function organize memory in
this way:

        +--------+----------------------
+-------------------------------------+
        |  size  |        data_ptr      | "/dev/
ttyACM0"                      |
        +--------+-----------+----------
+-------------------------------------+
                             |____________^

And cast initial address to _Opaque_string_type*.

     ... which is probably wrong, because normally a `struct string**'
would point to a `struct string*' which would in turn point to a
`struct string'.  Where is the `struct string'?

     Or perhaps you meant to cast to `_Opaque_string_type', that is,
to a one-level `struct string', and treat the first part of the buffer
as if it were a `struct string' instance?  That might or might not
work, depending on how big a `struct string' is, what elements it
contains, and what its alignment requirements are.

Unfortunately, with this approach the string length is limited by the
size of the buffer, and I must declare buffer explicit.

I read about function `alloca', but I can't use it in a function
parameter list.

I will improve this method, someone can suggest a trick to me?

     I can't figure out what you're trying to do.  You've got some
kind of "opaque" string library, but you "know" that simply throwing
its memory objects away without the library's knowledge is a valid
thing to do.  That's not very "opaque," is it?  It also makes me
wonder just what extra value this library adds beyond what C's own
strings (primitive as they are) provide.  Dynamic expansion as the
strings grow?  No, because then you couldn't just discard the struct
without leaking the dynamic memory.

     So, what are you trying to do?

Sorry, I make mistake.


typedef struct string* _Opaque_string_type;
typedef struct device* _Opaque_device_type;

Choice of names is demonstrative. However structures are not defined
in header files, so only public-access functions can know and manage
members.

extern _Opaque_string_type string_new(const char *, ...);
extern void string_delete(_Opaque_string_type);

extern _Device_type device_open(_Opaque_string_type p_path);

void foo(void)
{
_Opaque_string_type str;
_Opaque_device_type dev;

str = string_new("/dev/ttyACM0");
dev = device_open(str);
string_delete(str);
}

As you can see, I need a call to string_new to have a dynamic-
allocated string, and a call to string_delete to free it.

As show in the previous message I make a function like string_new
that
alloc the structure in a buffer provided by the caller, prototype is
like:

_Opaque_string_type string_new_at(void *buffer, size_t buffer_size,
const char *format, ...);

Internally this function look at buffer_size, to make sure buffer can
hold sizeof(struct string). Let me show how internal implementation
can be:

struct string {
size_t size;
char *data;
};

/* _Opaque_string_type is defined as a pointer to a struct string
variable */

_Opaque_string_type
string_new_at(void *buffer, size_t buffer_size,
const char *format, ...)
{
_Opaque_string_type str;

/* check buffer_size ... */

str = (_Opaque_string_type) buffer;

str->size = buffer_size - sizeof(struct string);

str->data = (char*) buffer + sizeof(struct string);

/* fill data ... */

return str;
}

Now we can use this function to create a struct string into a buffer,
So the above example change to:

void foo(void)
{
char my_buffer[64];
_Opaque_device_type dev;

dev = device_open(
string_new_at(my_buffer,
sizeof(my_buffer), "/dev/ttyACM0") );
}

What I'm finding is (miracle). A method to alloc struct string
in the stack like with alloca.

But alloca man page says that I can't use it in a function parameter
list
like:

memcpy(alloca(1024), source);

It's impossible to do?

Thanks for patience!

Christian.

 

[brncrs]

     The buffer itself is a problem: How should the caller create it?

Using a char[] array won't always work, because the struct you want
to store inside it might have a stricter alignment requirement than
the array -- and since the struct declaration is hidden, there is no
way for the caller to know what alignment is required.

Oh, but when the function `string_new_at' cast the pointer from void*
to `_Opaque_string_type' the address is not right aligned?

Example:

char buf[1024];
_Opaque_string_type s = string_new_at(buf, sizeof(buf), "A simple
string");

     What happens here if my_buffer[] is too small to hold both the
struct and the string?  Remember, the struct declaration is not
visible here, so the caller has no way to know how much memory
it needs.

Yes, I public a macro like OPAQUE_STRING_TYPE_SIZE. This macro expand
to a constant value
that is sizeof(struct string). Size is calculated with GNU Autoconf.

     Again, I'm forced to ask: What benefit does this "opaque string"
package provide, given that the way you want to use it requires you
to violate its opacity?

I think the advantage is that, forcing the application to use access
functions
I can modify internal string implementation without modify the code
that use _Opaque_string_type.

However, right or wrong, I am very (but very) curious. I'd like to
learn a trick to allocate static variables on-the-fly.
I think with assembly (perhaps in a macro) we can do anything, even
what I'm trying to do...

Again, Many thanks!

Christian.

 

[jameskuyper]

The buffer itself is a problem: How should the caller create it?
Using a char[] array won't always work, because the struct you want
to store inside it might have a stricter alignment requirement than
the array -- and since the struct declaration is hidden, there is no
way for the caller to know what alignment is required.

Oh, but when the function `string_new_at' cast the pointer from void*
to `_Opaque_string_type' the address is not right aligned?

Example:

char buf[1024];
_Opaque_string_type s = string_new_at(buf, sizeof(buf), "A simple
string");

No, the standard allows structs to have alignment requirements, and it
also allows buf to be allocated in a way that does not meet those
requirements.

 

[Eric des Courtis]

Hi list!
        I'm writing out a program where string's are handled by opaque
structure and each string variable is allocated/deallocated
dynamically
using functions like `string_new' and `string_delete'.

In some part of this program I need variables of this type
only to let an argument to a function, and alloc/dealloc this variable
manually don't make me very happy.

Example:

        typedef struct string* _Opaque_string_type;
        typedef struct device* _Opaque_device_type;

        ...

        extern _Opaque_string_type* string_new(const char *, ...);
        extern void string_delete(_Opaque_string_type *);

        extern _Device_type* device_open(_Opaque_string_type* p_path);

        ...

        void foo(void)
        {
                _Opaque_string_type str;
                _Opaque_device_type dev;

                str = string_new("/dev/ttyACM0");
                dev = device_open( str );
                string_delete(str);
        }

The best solution I got is to let to the `string_new' function the
address of a buffer where to store the variable:

Example:

        typedef struct string* _Opaque_string_type;
        typedef struct device* _Opaque_device_type;

        ...

        extern _Opaque_string_type* string_new(void *, size_t, const
char *, ...);

        extern _Device_type* device_open(_Opaque_string_type* p_path);

        ...

        extern _Device_type* device_open(_Opaque_string_type* p_path);

        void foo(void)
        {
                char buf[64];
                _DeviceType *dev = device_open( string_new(buf, sizeof
(buf), "/dev/ttyACM0") );
        }

So the string is stored in `buf' and I don't need to free them with
`string_delete'. Internally the string_new function organize memory in
this way:

        +--------+----------------------
+-------------------------------------+
        |  size  |        data_ptr      | "/dev/
ttyACM0"                      |
        +--------+-----------+----------
+-------------------------------------+
                             |____________^

And cast initial address to _Opaque_string_type*.

Unfortunately, with this approach the string length is limited by the
size of the buffer, and I must declare buffer explicit.

I read about function `alloca', but I can't use it in a function
parameter list.

I will improve this method, someone can suggest a trick to me?

Excuse me for horrible English language!

Thanks in advance!

Christian.

You are aware you are essentially rewriting a small part of Glib?

Why not use Glib?

Eric

 

[Beej Jorgensen]

- Everything works just fine, but perhaps a bit slower than
if the address were correctly aligned

I have another one along these lines (which happened a few years ago;
this is the best of my memory):

I was editing someone else's (I swear!) code and removed an unused local
variable. That's all. That's all I did.

What could possibly go wrong?

Suddenly simple tried-and-true functions that EVERYONE had been using
for the ENTIRE PROJECT without issue started returning incorrect results
in this subsystem. E.g. vectorAdd(v1, v2) would return bad numbers.
(Lots of zero vectors, IIRC.)

To make sure I wasn't dreaming, I put the variable back, and everything
started working again.

Some quick investigation showed that a char[] declared after the local
variable was being hand-packed with structs. Everything just happened
to work out until I misaligned it by removing the other variable.
Vectors stored inside the structs were operated upon as 128-bit values
by opcodes that demanded proper alignment, and they weren't getting it.

I couldn't figure out why the array wasn't of the struct type in the
first place, since that's all that was being stored in there. So I
changed it and it worked fine ever since.

-Beej

 

[brncrs]

Static variables are allocated before the program starts and

remain allocated until it exits. What you're talking about is
something else: A kind of dynamically-allocated variable that
behaves as if it were `auto'. There are ways to get such things --
we've already discussed C99's variable-length arrays, and the
non-standard alloca() function -- but neither of them solves the
alignment problem.

Oh, variable-length array's are similar to my idea, get the
_Opaque_string_type variable
live into the buffer. But I'm looking for something out of schema's
Maybe I never use the method in my program. Again: I am only curious.

what I would do is a function that return a pointer to a
_Opaques_string_type, that don't need to be freed
explicitly. Basically I will be able to do this:

extern int open_device(const _Opaque_string_type p_path);

...

open_device(string_new("/dev/ttyACM0"));

...

It's not important where variable is, on the stack or on the heap. Is
Important that variable is freed automatically.

I think I can make `string_new' remember the pointer, and free it in
the next call.
Another method can use a macro:

#define OPEN_DEVICE(P_PATH) do { _Opaque_string_type temp_string =
string_new(P_PATH); open_device(temp_string); string_delete
(temp_string); } while(0)

Of course, there are many other "simple" trick's, but I will discover
something of strange.
If we use assembly? I don't know "stack", "frame" and similar concepts
very well, but I think assembly have not limits. Right?

I suppose that a variable declaration in C, make the stack pointer
smaller to hold the variable. It's Right? We can use this approach?


Many thanks, get opinions/ideas/comments make me very happy!

Christian.