thread specific sys.stdout?

[aurora]

This may sound a little crazy. I capture the output of one class by
redirecting the sys.stdout. However the is another threading running at
the same time and occasionaly it output some messages to the redirected
sys.stdout irreleveant to the output I want to capture. Is there a way to
redirect output specific to some threads?

aurora

 

[Diez B. Roggisch]

This may sound a little crazy. I capture the output of one class by
redirecting the sys.stdout. However the is another threading running at
the same time and occasionaly it output some messages to the redirected
sys.stdout irreleveant to the output I want to capture. Is there a way to
redirect output specific to some threads?

You could replace sys.stdout by a class that splits the written text
depending on the current thread. It might look roughly like this:

class ThreadPrinter:
def __init__(self):
_.fhs = {}

def write(self, value):
f = _.fhs.get(threading.currentThread(),
open(get_some_nice_file_name(), "w")
f.write(value)
_.fhs[threading.currentThread()] = f

Now before starting your threads, replace sys.stdout with an instance of
ThreadPrinter:

sys.stdout = ThreadPrinter()

 

[Peter Hansen]

This may sound a little crazy. I capture the output of one class by
redirecting the sys.stdout. However the is another threading running at
the same time and occasionaly it output some messages to the redirected
sys.stdout irreleveant to the output I want to capture. Is there a way
to redirect output specific to some threads?

I don't know if there's a simpler way, but we once wrote a
redirector which checked threading.currentThread() to determine
whether a particular .write() call should be redirected or
just passsed through to the original output via sys.__stdout__.

Sorry, I don't have access to the code any more, but it shouldn't
be hard for you to reproduce.

-Peter

 

[Peter Hansen]

You could replace sys.stdout by a class that splits the written text
depending on the current thread. It might look roughly like this:

class ThreadPrinter:
def __init__(self):
_.fhs = {}

def write(self, value):
f = _.fhs.get(threading.currentThread(),
open(get_some_nice_file_name(), "w")
f.write(value)
_.fhs[threading.currentThread()] = f

Have you run this code? It looks to me suspiciously as
though it will raise an exception on the second write
call in any given thread, as the non-shortcircuiting call
to .get() tries to open the nice_file in write mode for
a second time.

Also, what's "_" supposed to be here? self?

-Peter

 

[Diez B. Roggisch]

Have you run this code? It looks to me suspiciously as
though it will raise an exception on the second write
call in any given thread, as the non-shortcircuiting call
to .get() tries to open the nice_file in write mode for
a second time.

Nope, didn't run it - and you are right of course. I should have said more
clearly that the code was untested - I thought that describing it as

"might look roughly like this"

would suffice.

Also, what's "_" supposed to be here?  self?

Yup it is - I use _ for self - I tried to adapt to the common standard for
the post, but failed in the middle of it. Sorry for the confusion.

 

[Peter Hansen]

Nope, didn't run it - and you are right of course. I should have said more
clearly that the code was untested - I thought that describing it as

"might look roughly like this"

would suffice.

That probably would have sufficed, but I'm one of those people
that tends not to read the documentation. I just jumped to
the code. ;-)

-Peter

 

[aurora]

This may sound a little crazy. I capture the output of one class by
redirecting the sys.stdout. However the is another threading running at
the same time and occasionaly it output some messages to the redirected
sys.stdout irreleveant to the output I want to capture. Is there a way
to
redirect output specific to some threads?

You could replace sys.stdout by a class that splits the written text
depending on the current thread. It might look roughly like this:

class ThreadPrinter:
def __init__(self):
_.fhs = {}

def write(self, value):
f = _.fhs.get(threading.currentThread(),
open(get_some_nice_file_name(), "w")
f.write(value)
_.fhs[threading.currentThread()] = f

Now before starting your threads, replace sys.stdout with an instance of
ThreadPrinter:

sys.stdout = ThreadPrinter()

Thanks this is a nice idea. I hope Python would actually support the '_'
syntax. The self really reduce readablity, especially if you have several
of them in one line.

 

[Peter Hansen]

.....

Thanks this is a nice idea. I hope Python would actually support the
'_' syntax. The self really reduce readablity, especially if you have
several of them in one line.

It does! One just has to be consistent within each function.
Diez changed the code from something like this:

def __init__(_):
_.fhs = {}

def write(_, value):
f = _.fhs.get(threading.currentThread(),
....

Some would argue that this is actually less readable, however,
since it uses punctuation instead of a word. If nothing else,
you run into a bit of a conflict between your own technique,
with "_", and the vast majority of the rest of the Python world,
which uses "self" exclusively, leading to situations like this
one...

(I think if I had a routine that really heavily used self,
to the obvious detriment of readability, and it wasn't clear
how else to improve it, I would use a local assignment at
the top to make a shorter name, perhaps "s", or even "_" --
but I wouldn't use the possibility of such a thing as a
justification for using _ everywhere.)

-Peter

 

[aurora]

It does! One just has to be consistent within each function.
Diez changed the code from something like this:

def __init__(_):
_.fhs = {}

def write(_, value):
f = _.fhs.get(threading.currentThread(),
...

Some would argue that this is actually less readable, however,
since it uses punctuation instead of a word. If nothing else,
you run into a bit of a conflict between your own technique,
with "_", and the vast majority of the rest of the Python world,
which uses "self" exclusively, leading to situations like this
one...

(I think if I had a routine that really heavily used self,
to the obvious detriment of readability, and it wasn't clear
how else to improve it, I would use a local assignment at
the top to make a shorter name, perhaps "s", or even "_" --
but I wouldn't use the possibility of such a thing as a
justification for using _ everywhere.)

-Peter

Didn't aware that _ itself is a valid identifier! True, you don't really
want to do things differently from convention. I'm just ranting about
the verbosity of self.

This doesn't take a complicated statement to make it really clumsy. Some
simple statement would look like this:

if self.max < self.list[self.index]:
self.max = self.list[self.index]:

Replacing self with _, depends on one's aesthetic, it could be ugly or
it could be cleaner. I like it that it is not a word and it does not
interfere with the keywords that's really relevant.

if _.max < _.list[_.index]:
_.max = _.list[_.index]:

Of couse I think this syntax the best:

if max < list[index]:
max = list[index]:

This remind me of those awful Hungarian notation.

aurora

 

[Alex Martelli]

Of couse I think this syntax the best:

if max < list[index]:
max = list[index]:

Just to ensure that the best approach,
self.max = max(self.max, self.list[index])
isn't available any more, _and_ you can't use the list built-in name any
more either? What a scoop!

This remind me of those awful Hungarian notation.

Explicit scope denotation, and Hungarian notation (which prefixes names
with type-connected prefix strings), have essentially nothing to do with
each other, of course. Making classes implicit scopes (like, say, C++,
but differently from, say, Modula-3) is simply a horrid mess, where you
can't use bare names safely without carefully studying all the internals
of all your ancestor classes... and if any such ancestor ever adds a
private name it can break every subclass which _did_ use bare names. A
bad idea even in a language where the compiler can find out statically
where every name comes from (because human readers can't), just as bad
as "from foo import *" in Python or "using namespace foo" in C++ except
that you can't avoid it by just eschewing one misdesigned construct.

In a language where even the compiler _cannot_ tell statically which
bare names come from where (except for functions' locals), criticizing
the language design choice of _not_ making classes into implcit scopes
doesn't even _verge_ on the ridiculous -- it plunges right deep into it.


Alex

 

[Alex Martelli]

This may sound a little crazy. I capture the output of one class by
redirecting the sys.stdout. However the is another threading running at
the same time and occasionaly it output some messages to the redirected
sys.stdout irreleveant to the output I want to capture. Is there a way to
redirect output specific to some threads?

You could replace sys.stdout by a class that splits the written text
depending on the current thread. It might look roughly like this:

class ThreadPrinter:
def __init__(self):
_.fhs = {}

def write(self, value):
f = _.fhs.get(threading.currentThread(),
open(get_some_nice_file_name(), "w")
f.write(value)
_.fhs[threading.currentThread()] = f

Not a bad general idea, but you need a better implementation of the
"thread-local storage" design pattern than just a bare dictionary like
this 'fhs' dict. In Python 2.4, threading.local gives you such an
implementation. If you need to work in Python 2.3, it's more work, but
there are cookbook recipes (on Activestate's site) which can help.


Alex

 

[aurora]

Of couse I think this syntax the best:

if max < list[index]:
max = list[index]:

Just to ensure that the best approach,
self.max = max(self.max, self.list[index])
isn't available any more, _and_ you can't use the list built-in name any
more either? What a scoop!

This remind me of those awful Hungarian notation.

Explicit scope denotation, and Hungarian notation (which prefixes names
with type-connected prefix strings), have essentially nothing to do with
each other, of course. Making classes implicit scopes (like, say, C++,
but differently from, say, Modula-3) is simply a horrid mess, where you
can't use bare names safely without carefully studying all the internals
of all your ancestor classes... and if any such ancestor ever adds a
private name it can break every subclass which _did_ use bare names. A
bad idea even in a language where the compiler can find out statically
where every name comes from (because human readers can't), just as bad
as "from foo import *" in Python or "using namespace foo" in C++ except
that you can't avoid it by just eschewing one misdesigned construct.

In a language where even the compiler _cannot_ tell statically which
bare names come from where (except for functions' locals), criticizing
the language design choice of _not_ making classes into implcit scopes
doesn't even _verge_ on the ridiculous -- it plunges right deep into it.


Alex

I'm not making any serious criticism on the language or how it should do
name binding. I'm ranting about having to attach a prefix to names often
make simple things look complicated. An annoyance when it has to be done
very often. You got the point?

 

[Elbert Lev]

It does! One just has to be consistent within each function.
Diez changed the code from something like this:

def __init__(_):
_.fhs = {}

def write(_, value):
f = _.fhs.get(threading.currentThread(),
...

Some would argue that this is actually less readable, however,
since it uses punctuation instead of a word. If nothing else,
you run into a bit of a conflict between your own technique,
with "_", and the vast majority of the rest of the Python world,
which uses "self" exclusively, leading to situations like this
one...

(I think if I had a routine that really heavily used self,
to the obvious detriment of readability, and it wasn't clear
how else to improve it, I would use a local assignment at
the top to make a shorter name, perhaps "s", or even "_" --
but I wouldn't use the possibility of such a thing as a
justification for using _ everywhere.)

-Peter

Didn't aware that _ itself is a valid identifier! True, you don't really
want to do things differently from convention. I'm just ranting about
the verbosity of self.

This doesn't take a complicated statement to make it really clumsy. Some
simple statement would look like this:

if self.max < self.list[self.index]:
self.max = self.list[self.index]:

Replacing self with _, depends on one's aesthetic, it could be ugly or
it could be cleaner. I like it that it is not a word and it does not
interfere with the keywords that's really relevant.

if _.max < _.list[_.index]:
_.max = _.list[_.index]:

Of couse I think this syntax the best:

if max < list[index]:
max = list[index]:

This remind me of those awful Hungarian notation.

aurora

Please do not do this, please...
Recently I was working with a modules written by a programmer,
who used "_" instead of "self".
This was so "not in line" with the rest of the system,
that it took me extra hour or two to get accustomed
and switch back and force (self in "standard" modules and _ in "economical").

 

[Ville Vainio]

aurora> I'm not making any serious criticism on the language or
aurora> how it should do name binding. I'm ranting about having to
aurora> attach a prefix to names often make simple things look
aurora> complicated. An annoyance when it has to be done very
aurora> often. You got the point?

Why don't you write a preprocessor that converts every .foo to
self.foo?

 

[Alex Martelli]

I'm not making any serious criticism on the language or how it should do
name binding. I'm ranting about having to attach a prefix to names often
make simple things look complicated. An annoyance when it has to be done
very often. You got the point?

No. If you had to "attach a prefix", it might perhaps be a problem.
But, in Python, you don't have to do any such thing. You use bare names
for locals, globals, and built-ins, and compound names for attributes of
objects. How can it "make simple things look complicated" to use a
compound name for an object attribute? It indicates exactly what's
going on: you're accessing or rebinding an attribute -- no more, no
less. Just like, e.g., x is how you refer to an item of x, so is
x.foo how you refer to an attribute of x.

If what you mean to say (as opposed to what you actually wrote) is to
support some kind of 'with' or 'using' statement, such as:

with glek[idx]:
.total += .partial
.partial = current_time
.laps += 1

so that several accesses to, and rebindings of, attributes of the same
object, can be compactly and conveniently indicated, that, of course, is
quite a different issue, which has often been discussed in the past,
both here and on python-dev. Yet there isn't even a PEP for it, yet, I
believe -- a bad sign: it suggests nobody is keen enough on it to
summarize the case for and against and nail down the specs. I guess the
problem is that if you see this as equivalent to, say:

_ = glek[idx]:
_.total += _.partial
_.partial = current_time
_.laps += 1

it's not all that clear that the with notation is buying you all that
much -- visually distinguishing that current_time is not an attribute
name is about as hard in both cases, for example, maybe a bit harder if
the hypothetical new construct were in use... and without said
hypothetical new construct you have a better chance to make your code
clearer by using a visually distinguished name rather than a notation
which appears to try to HIDE the crucial issue of which name are bare
ones, and which names are compound!

Richer semantics for 'with'/'using' are of course possible, but harder
to pin down in detail, and even more controversial. Still, until
somebody DOES care enough, for or against, to come up with a PEP, rather
than rants on this NG, nothing much is going to happen re this idea.


Alex

 

[Bengt Richter]

I'm not making any serious criticism on the language or how it should do
name binding. I'm ranting about having to attach a prefix to names often
make simple things look complicated. An annoyance when it has to be done
very often. You got the point?

No. If you had to "attach a prefix", it might perhaps be a problem.
But, in Python, you don't have to do any such thing. You use bare names
for locals, globals, and built-ins, and compound names for attributes of
objects. How can it "make simple things look complicated" to use a
compound name for an object attribute? It indicates exactly what's
going on: you're accessing or rebinding an attribute -- no more, no
less. Just like, e.g., x is how you refer to an item of x, so is
x.foo how you refer to an attribute of x.

If what you mean to say (as opposed to what you actually wrote) is to
support some kind of 'with' or 'using' statement, such as:

with glek[idx]:
.total += .partial
.partial = current_time
.laps += 1

so that several accesses to, and rebindings of, attributes of the same
object, can be compactly and conveniently indicated, that, of course, is
quite a different issue, which has often been discussed in the past,
both here and on python-dev. Yet there isn't even a PEP for it, yet, I
believe -- a bad sign: it suggests nobody is keen enough on it to
summarize the case for and against and nail down the specs. I guess the
problem is that if you see this as equivalent to, say:

_ = glek[idx]:
_.total += _.partial
_.partial = current_time
_.laps += 1

it's not all that clear that the with notation is buying you all that
much -- visually distinguishing that current_time is not an attribute
name is about as hard in both cases, for example, maybe a bit harder if
the hypothetical new construct were in use... and without said
hypothetical new construct you have a better chance to make your code
clearer by using a visually distinguished name rather than a notation
which appears to try to HIDE the crucial issue of which name are bare
ones, and which names are compound!

Richer semantics for 'with'/'using' are of course possible, but harder
to pin down in detail, and even more controversial. Still, until
somebody DOES care enough, for or against, to come up with a PEP, rather
than rants on this NG, nothing much is going to happen re this idea.

It occurs to me that "with obj_expr:" selects a specific name space and
access mechanism (attribute access, which looks through base classes for
decriptors etc), whereas one could expand the concept of what the leading
dot means as an access operation. E.g., the normal default spelled out might be

with(glek[idx], dotop=with.attr): # with.attr would be a get/set pair for attributes
.total += .partial
...

Or you could limit yourself to an instance dict by alternate meaning
for '.' access, e.g.,

with(vars(inst), dotop=with.item): # with.item would be get/set for items
.total += .partial # meaning v=vars(inst); v['total'] += v['partial']; del v
...

or you could specify a list of spaces to chase in explicit order, e.g.,
object attribute spaces:

with(a,b,c):
x = .xxx # a.xxx if avail, else b.xxx, else c.xxx, else AttributeError

Anyway, you get the general idea. A way to spell out access to any alternative
name space or search path through several. ;-)

Regards,
Bengt Richter