Putting function references in a Queue

[Richard Townsend]

I've been experimenting putting a reference to a function into a Queue
object and was wondering what actually gets put in the Queue - is it the
function's code object?

If I read from the Queue in a different module, it appears that I don't
need to import the module that defines the function - or any module that it
uses - is this generally true, or are there some conditions to be aware of?

The scenario I'm working on has child threads doing some tasks and then
sending back tuples (done, function, args, kwargs) via the Queue, to be
called in the main thread. The Python code is ultimately embedded in a
C/Motif app.

 

[tiissa]

I've been experimenting putting a reference to a function into a Queue
object and was wondering what actually gets put in the Queue - is it the
function's code object?

It would simply be the entire function object (unless you choose it
otherwise).

If I read from the Queue in a different module, it appears that I don't
need to import the module that defines the function - or any module that it
uses - is this generally true, or are there some conditions to be aware of?

Functions are objects that carry around their code but also their global
namespace (including modules) and some other things.

With a function f, try for instance 'dir(f)' and 'f.func_globals'.

 

[Benjamin Niemann]

I've been experimenting putting a reference to a function into a Queue
object and was wondering what actually gets put in the Queue - is it the
function's code object?

No, it's justa referenceto the function object.

If I read from the Queue in a different module, it appears that I don't
need to import the module that defines the function - or any module that
it uses - is this generally true, or are there some conditions to be aware
of?

There function reference is sufficient to call the function, there's not
need to import the module containing the function. It has already been
imported (otherwise the function wouldn't be there) and the function has a
reference to this module in order to resolve its global (module-level)
references.

The scenario I'm working on has child threads doing some tasks and then
sending back tuples (done, function, args, kwargs) via the Queue, to be
called in the main thread. The Python code is ultimately embedded in a
C/Motif app.

As long as you take care of the usual threading issues (concurrent access to
shared objects guarded by semaphores etc.), there should not be any greater
problems.
The function will be executed in the thread that is called it of course (and
not in the thread that sent the reference to it).

 

[Bryan Olson]

> I've been experimenting putting a reference to a function into a Queue
> object and was wondering what actually gets put in the Queue - is it the
> function's code object?

It's a Python-internal-reference-thingy.

> If I read from the Queue in a different module, it appears that I don't
> need to import the module that defines the function - or any module that it
> uses - is this generally true, or are there some conditions to be

aware of?

Yes, generally true. It has to be implemented somewhere in the
process; the reference-thingy knows where.

> The scenario I'm working on has child threads doing some tasks and then
> sending back tuples (done, function, args, kwargs) via the Queue, to be
> called in the main thread. The Python code is ultimately embedded in a
> C/Motif app.

I think that's a good thing to do. The tricky part is getting an
event loop to wait on both the queue and other kinds of events.
Periodic polling works, but kind of sucks.

What's the 'done' argument? A lock maybe?

 

[Richard Townsend]

I think that's a good thing to do. The tricky part is getting an
event loop to wait on both the queue and other kinds of events.
Periodic polling works, but kind of sucks.

What's the 'done' argument? A lock maybe?

'done' is just a boolean for the child thread to indicate it has finished.
The main thread keeps a dictionary of references to the child threads so
the app can send then 'abort' messages via their input Queues. When a child
thread sets 'done' to True, the main thread deletes its reference from the
dictionary.

Periodic polling triggered by a timer in the X-app main loop was the only
way I could think of checking for feedback from the child threads, not very
neat though...

Thanks to everyone for their input!