copy/pass a va_list to another thread

C

Christopher

Working on a pre-existing interface that I am not allowed to change, which uses a va_list argument. The method takes way too long to execute and is a canidate for splitting off into another thread. However, I am not sure how I can safely copy the va_list without actually processing its real components. The processing into its real components is a good chunk of what is taking too long.

Any ideas on how to safely pass a varibale argument list to another thread to be processed?
 
A

Alf P. Steinbach

Working on a pre-existing interface that I am not allowed to change,
which uses a va_list argument. The method takes way too long to execute
and is a canidate for splitting off into another thread. However, I am
not sure how I can safely copy the va_list without actually processing
its real components. The processing into its real components is a good
chunk of what is taking too long.

If you're talking about a stack frame in the megabytes or gigabytes
range, then the function (and most likely much more) needs redesign.

Otherwise you have most probably misunderstood something.

For more concrete to-the-point help it is a good idea to copy and paste
into your Usenet posting working code for a minimal but complete example
that actually illustrates the problem.

Any ideas on how to safely pass a varibale argument list to another
thread to be processed?

You have already stated that copying takes too long.

And you can't pass a reference or pointer to it without letting the
original thread wait.

So at first glance the problem is insoluble.

However, it is almost certain that you have misunderstood something.

Just post real code instead of a vague and most likely incorrect
description.


Cheers & hth.,

- Alf
 
B

BGB

Working on a pre-existing interface that I am not allowed to change, which uses a va_list argument. The method takes way too long to execute and is a canidate for splitting off into another thread. However, I am not sure how I can safely copy the va_list without actually processing its real components. The processing into its real components is a good chunk of what is taking too long.

Any ideas on how to safely pass a varibale argument list to another thread to be processed?

no, va_list can't be safely passed (given they depend on the callers'
stack frame, which is thread-local).

best option I know of:
read arguments from va_list into some other format (such as an array);
pass a copy of this data instead.

granted, this requires some means to know the layout of the list in
advance (number and types of arguments), which is presumably available
somehow?
 
C

Christopher

You have already stated that copying takes too long.

And you can't pass a reference or pointer to it without letting the
original thread wait.

So at first glance the problem is insoluble.

Maybe. I am hoping for some boost or C++ standard mechanism, which I haven't heard of yet, that solves this problem that I presume many developers have run across, when dealing with a function that takes a variable argument list parameter.
Just post real code instead of a vague and most likely incorrect
description.

I think you are reading to much into it and want to look at the design for a fix. I don't blame you. I already accept that the design is flawed, the code sucks, and I'd never use a variable argument list in the first place. Such is often life when dealing with other people's real life code and manager's rules and time allotments vs academic examples. Given those constants,it doesn't matter what the context is, but here:

// Creates a database log record
// \param sprintf like message string including placeholders
// \param ... - takes tuples
// 1- string; name of argument
// 2- string; formatting code string
// 3- any type; identified by previous formatting code
// <list of 500 formatting codes and argument types>
void SomeFunction(std::wstring message, ...) // Cannot change
{
va_list va; // May change anything here down
va_start(va, x);

SomeDataStructure myOutput; // Contains arguments for a database call
// to create a log record
// which is also crappy design
// with tons of params
// and encoded, formatted, strings

// 4000 lines of parsing custom format codes and arguments,
// converting to types
// using those types to build strings
// ala sprintf
// With a dump truck of business rules applied to string encoding
// AKA long and tedious stuff that I want to do on another thread
// Including actually making a call to write to database which is
// also time consuming
//
// i.e. This is a logging type method and should return immediatly

va_end(va);
}

The actual conversion from unknown types into usuable typed objects using format codes in strings is what is taking a long time. I really don't think I mentioned copying as you say. There is no copying yet. I envision copyingas a means to pass them to another thread. As obviously I can't give something to another thread from the stack without blocking the first thread or they will no longer be valid.

The problem itself is how to copy something of unknown type and length fromthe "stack pointer?" given by va_list, without actually processing the list to find the types first.

performance analysis revealed that the processing of the list, just to extract the types and use them to build the output to make the database call with, took longer than the database call itself!
 
A

Alf P. Steinbach

On 08.03.2012 20:54, Christopher wrote:
[snip text]
// Creates a database log record
// \param sprintf like message string including placeholders
// \param ... - takes tuples
// 1- string; name of argument
// 2- string; formatting code string
// 3- any type; identified by previous formatting code
//<list of 500 formatting codes and argument types>
void SomeFunction(std::wstring message, ...) // Cannot change
{
va_list va; // May change anything here down
va_start(va, x);

SomeDataStructure myOutput; // Contains arguments for a database call
// to create a log record
// which is also crappy design
// with tons of params
// and encoded, formatted, strings

// 4000 lines of parsing custom format codes and arguments,
// converting to types
// using those types to build strings
// ala sprintf
// With a dump truck of business rules applied to string encoding
// AKA long and tedious stuff that I want to do on another thread
// Including actually making a call to write to database which is
// also time consuming
//
// i.e. This is a logging type method and should return immediatly

va_end(va);
}

[snip text]

First, PLEASE DO NOT POST WITH QUOTED PRINTABLE FORMAT.

It really really sucks.

I see you're posting via Google. They have tried umpteen times to kill
Usenet. Why they're still bothering to kill a dead horse is beyond me,
perhaps the Google people doing Google Groups stuff now are just grossly
technically incompetent and also unaware of history, but anyway, please
don't post with quoted printable, like, please do not go naked and
unwashed into a restaurant or, for that matter, bookstore.

That said (phew!), your task, except for error handling, is easy but
tedious.

You need to parse the format string and retrieve the original argument
values. Possibly you can get by with just storing a string
representation of each argument value. Otherwise, a heterogeneous list
is easy to do in C++, check out boost::variant and boost::any. Pass the
argument collection etc. to the thread.

Error handling problem: how to notify the calling code, which presumably
is unaware of the asynchronous nature of the call, of errors and failures.

It might be a Good Idea(TM) to really think about that before embarking
on the easy but very tedious task outlined above. <g>


Cheers & hth.,

- Alf
 
C

Christopher

First, PLEASE DO NOT POST WITH QUOTED PRINTABLE FORMAT.

It really really sucks.

I see you're posting via Google. They have tried umpteen times to kill
Usenet. Why they're still bothering to kill a dead horse is beyond me,
perhaps the Google people doing Google Groups stuff now are just grossly
technically incompetent and also unaware of history, but anyway, please
don't post with quoted printable, like, please do not go naked and
unwashed into a restaurant or, for that matter, bookstore.

I apologize as this is inevitably also in the format you spoke of, as I have yet to find a means to access usenet without paying an extra $10/20 a month and see no means of configuring google to not be retarded. I also know little about said format, although I looked it up. I assume it makes jumbled reading on your side that expects ASCII? Is there a known free alternative that you might suggest?
 
R

red floyd

I apologize as this is inevitably also in the format you spoke of,
as I have yet to find a means to access usenet without paying an
extra $10/20 a month and see no means of configuring google to not
be retarded. I also know little about said format, although I looked
it up. I assume it makes jumbled reading on your side that expects
ASCII? Is there a known free alternative that you might suggest?

Eternal September. Go to www.eternal-september.org for info.
 
J

Jorgen Grahn

Not history -- Usenet conventions are the same today, at least in the
groups I frequent. "Ignorant of the culture they're targeting" is more
like it.
I apologize as this is inevitably also in the format you spoke of,
as I have yet to find a means to access usenet without paying an extra
$10/20 a month and see no means of configuring google to not be
retarded. I also know little about said format, although I looked it
up. I assume it makes jumbled reading on your side that expects ASCII?

I assume this is the new new Google Groups interface that they're
rolling out now.

Quoted-printable doesn't seem to create a problem on my side in your
posting; perhaps there was nothing that needed quoting?

What's worse, Christopher: your whole paragraph was just one single
line, 300 characters wide or so. Is that something you can influence?
By manual line breaking or by configuring Google Groups (a checkbox
"don't break Usenet netiquette" perhaps?)

If not, I will have to start ignoring posts from Google Groups, no
matter how interesting. I always disliked when people did that (or
pretended to) because postings from there were not usually less
readable than others[1]. This, however, is just too broken to deal
with.

/Jorgen

[1] Except the threading fiasco which was in effect for a while.
That was also destructive.
 
M

Miles Bader

Alf P. Steinbach said:
First, PLEASE DO NOT POST WITH QUOTED PRINTABLE FORMAT.

It really really sucks.

Why?

Are there people out there still using newsreaders from 1987...?

-miles
 
A

Alf P. Steinbach

Why?

Are there people out there still using newsreaders from 1987...?

On the contrary, quoted printable was designed to be displayable by
really old software and transmittable over really old networks, software
and networks unable to handle 8-bit character codes (i.e., using only 7
bits per character).

But there are people using e.g. Thunderbird as newsreader, and as of
2012 Thunderbird treats a quoted printable paragraph as one long single
line.

So you were wrong about the implicit contention that quoted printable is
anything modern (it isn't, it's for really old stuff, archaic stuff),
and you were wrong newsreaders that don't handle it well being old (the
latest version of Thunderbird is not very old).

However, that ungood treatment of this archaic 7-bit format in one of
the few surviving newsreaders (Thunderbird) is just one example of how
quoted printable really really sucks.


Cheers & hth.,

- Alf
 
M

Miles Bader

Alf P. Steinbach said:
On the contrary, quoted printable was designed to be displayable by
really old software and transmittable over really old networks,
software and networks unable to handle 8-bit character codes (i.e.,
using only 7 bits per character).

Quoted-printable is part of MIME, and any decent modern mail/news
reader supports MIME (including thunderbird, if my googling is
correct). [In the 1980s MIME didn't exist, of course, and widespread
adoption is more a late-90s thing.]

You're probably correct that quoted-printable is often _unnecessary_
with modern infrastructure, but that has nothing to do with the level
of support for MIME (which obviously addresses many more issues than
making content 7-bit safe), and thus with the level of support for
quoted-printable. Whether or not it's necessary, quoted-printable
should still work fine.

7-bit links: olde thing. MIME: newish thing.
But there are people using e.g. Thunderbird as newsreader, and as of
2012 Thunderbird treats a quoted printable paragraph as one long
single line.

Quoted-printable is a _transfer encoding_, and shouldn't affect the way
the article content is displayed by your newsreader.... and indeed,
looking at the article you're complaining about, the problem isn't with
quoted-printable at all, it's that the original article doesn't have
any newlines in its paragraphs.

So your anger should really be addressed at the fact that the sender's
software didn't wrap lines, not that it used quoted-printable to encode
those long lines.

Here: as a test, I'll try to make sure my newsreader uses
quoted-printable, and you can tell me if you have problems reading it
or not. My lines are properly wrapped.

-miles
 
M

Miles Bader

p.s.: my attempt to force quoted-printable didn't work ... :[

(my newsreader has a "preview encoded form" command which claimed it
was going to use QP, but obviously it wasn't accurate... oh well)

-miles
 
J

Jorgen Grahn

Alf P. Steinbach said:
On the contrary, quoted printable was designed to be displayable by
really old software and transmittable over really old networks,
software and networks unable to handle 8-bit character codes (i.e.,
using only 7 bits per character).

Quoted-printable is part of MIME, and any decent modern mail/news
reader supports MIME (including thunderbird, if my googling is
correct). [In the 1980s MIME didn't exist, of course, and widespread
adoption is more a late-90s thing.]

You're probably correct that quoted-printable is often _unnecessary_
with modern infrastructure, but that has nothing to do with the level
of support for MIME (which obviously addresses many more issues than
making content 7-bit safe), and thus with the level of support for
quoted-printable. Whether or not it's necessary, quoted-printable
should still work fine.

7-bit links: olde thing. MIME: newish thing.

MIME over (what may be) 7-bit links: quoted-printable.

Real life is a bit more complicated than this. The NetNews RFCs
weren't updated for many years. Between 1987 and 2005, there was no
standard for NetNews messages using MIME. So as far as I understand
it, there were more or less two camps:

- Stick to the RFCs for compatibility.
- Let's go crazy with everything that MIME allows for mail,
including HTML, images, OpenPGP signatures ...

Most serious posters chose the first option -- supplemented with Henry
Spencer's "Son of 1036" RFC draft, which also codifies things like
line lengths, ">" for quoting, "Re:" for responses and "-- " for
signature delimiter.

That's why I restrict my postings to ASCII, even though that means
misspelling my name. (In the now mostly dead .se part of Usenet the
convention was to violate the RFCs by sending 8-bit Latin-1, not
necessarily with any MIME info.)

The new set of RFCs (see http://tools.ietf.org/wg/usefor/) seem to
allow most of MIME in NetNews postings. They have
draft-ietf-usefor-useage-01.txt which seem to recommend 8-bit whenever
possible. I seem to recall that earlier drafts tried to nudge everyone
into using 8-bit UTF-8, but that USEFOR couldn't reach an agreement on
that. (MIME predates UTF-8.)

/Jorgen
 
A

Alf P. Steinbach

Alf P. Steinbach said:
On the contrary, quoted printable was designed to be displayable by
really old software and transmittable over really old networks,
software and networks unable to handle 8-bit character codes (i.e.,
using only 7 bits per character).

Quoted-printable is part of MIME, and any decent modern mail/news
reader supports MIME (including thunderbird, if my googling is
correct). [In the 1980s MIME didn't exist, of course, and widespread
adoption is more a late-90s thing.]

You're probably correct that quoted-printable is often _unnecessary_
with modern infrastructure, but that has nothing to do with the level
of support for MIME (which obviously addresses many more issues than
making content 7-bit safe), and thus with the level of support for
quoted-printable. Whether or not it's necessary, quoted-printable
should still work fine.

7-bit links: olde thing. MIME: newish thing.
But there are people using e.g. Thunderbird as newsreader, and as of
2012 Thunderbird treats a quoted printable paragraph as one long
single line.

Quoted-printable is a _transfer encoding_, and shouldn't affect the way
the article content is displayed by your newsreader.... and indeed,
looking at the article you're complaining about, the problem isn't with
quoted-printable at all, it's that the original article doesn't have
any newlines in its paragraphs.

That is incorrect.

So your anger

That is ad hominem (a personal attack).

should really be addressed at the fact that the sender's
software didn't wrap lines, not that it used quoted-printable to encode
those long lines.

That is dumb.

Here: as a test, I'll try to make sure my newsreader uses
quoted-printable, and you can tell me if you have problems reading it
or not. My lines are properly wrapped.

You are posting with text/plain, not quoted printable, and your
paragraphs are not paragraphs but individual lines.


Cheers & hth.,

- Alf
 
S

Stuart Redmann

On 8 Mrz., 20:54, Christopher wrote:

[snip]
Such is often life when dealing with other people's real life code
and manager's rules and time allotments vs academic examples. Given
those constants, it doesn't matter what the context is, but here:

// Creates a database log record
// \param sprintf like message string including placeholders
// \param ... - takes tuples
// 1- string; name of argument
// 2- string; formatting code string
// 3- any type; identified by previous formatting code
// <list of 500 formatting codes and argument types>
void SomeFunction(std::wstring message, ...) // Cannot change
{
va_list va; // May change anything here down
va_start(va, x);
[snip]
// 4000 lines of parsing custom format codes and arguments,
// converting to types using those types to build strings
// ala sprintf [snip]
va_end(va);

}

The actual conversion from unknown types into usuable typed
objects using format codes in strings is what is taking a
long time. [snip]
performance analysis revealed that the processing of the list,
just to extract the types and use them to build the output to
make the database call with, took longer than the database call
itself!

As far as I understand, you don't want to process the command
in the thread that called SomeFunction but in a worker thread,
but you don't know how many parameters must be copied to the
worker thread without parsing the parameter "message".

This could be solved in an unsafe way by copying "suffiently much"
stack parameters to some heap location where the worker thread can
access it.

However, most likely it will be the case that some parameters
of SomeFunction are only valid as long as SomeFunction runs. So you
cannot simply copy the parameters and return from SomeFunctions
because your worker thread will then operate on invalid data.

I guess you will have to bite the bullet and try to tell your
management that your function needs to be redesigned. Hard luck!

Regards,
Stuart
 
8

88888 Dihedral

Christopheræ–¼ 2012å¹´3月9日星期五UTC+8上åˆ3時54分06秒寫é“:
Maybe. I am hoping for some boost or C++ standard mechanism, which I haven't heard of yet, that solves this problem that I presume many developers have run across, when dealing with a function that takes a variable argumentlist parameter.


I think you are reading to much into it and want to look at the design for a fix. I don't blame you. I already accept that the design is flawed, thecode sucks, and I'd never use a variable argument list in the first place.Such is often life when dealing with other people's real life code and manager's rules and time allotments vs academic examples. Given those constants, it doesn't matter what the context is, but here:

// Creates a database log record
// \param sprintf like message string including placeholders
// \param ... - takes tuples
// 1- string; name of argument
// 2- string; formatting code string
// 3- any type; identified by previous formatting code
// <list of 500 formatting codes and argument types>
void SomeFunction(std::wstring message, ...) // Cannot change
{
va_list va; // May change anything here down
va_start(va, x);

SomeDataStructure myOutput; // Contains arguments for a database call
// to create a log record
// which is also crappy design
// with tons of params
// and encoded, formatted, strings

// 4000 lines of parsing custom format codes and arguments,
// converting to types
// using those types to build strings
// ala sprintf
// With a dump truck of business rules applied to string encoding
// AKA long and tedious stuff that I want to do on another thread
// Including actually making a call to write to database which is
// also time consuming
//
// i.e. This is a logging type method and should return immediatly

va_end(va);
}

The actual conversion from unknown types into usuable typed objects usingformat codes in strings is what is taking a long time. I really don't think I mentioned copying as you say. There is no copying yet. I envision copying as a means to pass them to another thread. As obviously I can't give something to another thread from the stack without blocking the first thread or they will no longer be valid.

The problem itself is how to copy something of unknown type and length from the "stack pointer?" given by va_list, without actually processing the list to find the types first.

performance analysis revealed that the processing of the list, just to extract the types and use them to build the output to make the database call with, took longer than the database call itself!



Christopheræ–¼ 2012å¹´3月9日星期五UTC+8上åˆ3時54分06秒寫é“:
Maybe. I am hoping for some boost or C++ standard mechanism, which I haven't heard of yet, that solves this problem that I presume many developers have run across, when dealing with a function that takes a variable argumentlist parameter.


I think you are reading to much into it and want to look at the design for a fix. I don't blame you. I already accept that the design is flawed, thecode sucks, and I'd never use a variable argument list in the first place.Such is often life when dealing with other people's real life code and manager's rules and time allotments vs academic examples. Given those constants, it doesn't matter what the context is, but here:

// Creates a database log record
// \param sprintf like message string including placeholders
// \param ... - takes tuples
// 1- string; name of argument
// 2- string; formatting code string
// 3- any type; identified by previous formatting code
// <list of 500 formatting codes and argument types>
void SomeFunction(std::wstring message, ...) // Cannot change
{
va_list va; // May change anything here down
va_start(va, x);

SomeDataStructure myOutput; // Contains arguments for a database call
// to create a log record
// which is also crappy design
// with tons of params
// and encoded, formatted, strings

// 4000 lines of parsing custom format codes and arguments,
// converting to types
// using those types to build strings
// ala sprintf
// With a dump truck of business rules applied to string encoding
// AKA long and tedious stuff that I want to do on another thread
// Including actually making a call to write to database which is
// also time consuming
//
// i.e. This is a logging type method and should return immediatly

va_end(va);
}

The actual conversion from unknown types into usuable typed objects usingformat codes in strings is what is taking a long time. I really don't think I mentioned copying as you say. There is no copying yet. I envision copying as a means to pass them to another thread. As obviously I can't give something to another thread from the stack without blocking the first thread or they will no longer be valid.

The problem itself is how to copy something of unknown type and length from the "stack pointer?" given by va_list, without actually processing the list to find the types first.

performance analysis revealed that the processing of the list, just to extract the types and usfuncte them to build the output to make the database call with, took longer than the database call itself!


I don't like this tyle of coding, and I'll suggest to use an input vector
to be passed to a function thus that all parameters are received in
the callee function to start the action.

Well, to pass by register does matter a lot in calling work-horse level
actions.
 
M

Mike McCarty

However, that ungood treatment of this archaic 7-bit format in one of
the few surviving newsreaders (Thunderbird) is just one example of how
quoted printable really really sucks.

I've never used it but it sounds like Thunderbird is what really really sucks,
being unable to reasonably display an "archaic" format like
quoted-printable.

Word-wrap is hardly a novel concept. I was using newsreaders in 1992 that
were capable of it (I know because people bitched about this problem then too).
 
J

Jorgen Grahn

On 10.03.2012 00:40, Miles Bader wrote: ....

That is incorrect.

How? There may be other problems with the posting
<7509868.1455.1331236446852.JavaMail.geo-discussion-forums@ynjd19>
but the overlong lines was the immediate problem I saw. Perhaps your reader
fixes such things for you?

/Jorgen
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,776
Messages
2,569,602
Members
45,184
Latest member
ZNOChrista

Latest Threads

Top