thread specific sys.stdout?

Discussion in 'Python' started by aurora, Sep 15, 2004.

  1. aurora

    aurora Guest

    This may sound a little crazy. I capture the output of one class by
    redirecting the sys.stdout. However the is another threading running at
    the same time and occasionaly it output some messages to the redirected
    sys.stdout irreleveant to the output I want to capture. Is there a way to
    redirect output specific to some threads?

    aurora
    aurora, Sep 15, 2004
    #1
    1. Advertising

  2. aurora wrote:

    > This may sound a little crazy. I capture the output of one class by
    > redirecting the sys.stdout. However the is another threading running at
    > the same time and occasionaly it output some messages to the redirected
    > sys.stdout irreleveant to the output I want to capture. Is there a way to
    > redirect output specific to some threads?


    You could replace sys.stdout by a class that splits the written text
    depending on the current thread. It might look roughly like this:

    class ThreadPrinter:
    def __init__(self):
    _.fhs = {}

    def write(self, value):
    f = _.fhs.get(threading.currentThread(),
    open(get_some_nice_file_name(), "w")
    f.write(value)
    _.fhs[threading.currentThread()] = f

    Now before starting your threads, replace sys.stdout with an instance of
    ThreadPrinter:

    sys.stdout = ThreadPrinter()


    --
    Regards,

    Diez B. Roggisch
    Diez B. Roggisch, Sep 15, 2004
    #2
    1. Advertising

  3. aurora

    Peter Hansen Guest

    aurora wrote:

    > This may sound a little crazy. I capture the output of one class by
    > redirecting the sys.stdout. However the is another threading running at
    > the same time and occasionaly it output some messages to the redirected
    > sys.stdout irreleveant to the output I want to capture. Is there a way
    > to redirect output specific to some threads?


    I don't know if there's a simpler way, but we once wrote a
    redirector which checked threading.currentThread() to determine
    whether a particular .write() call should be redirected or
    just passsed through to the original output via sys.__stdout__.

    Sorry, I don't have access to the code any more, but it shouldn't
    be hard for you to reproduce.

    -Peter
    Peter Hansen, Sep 15, 2004
    #3
  4. aurora

    Peter Hansen Guest

    Diez B. Roggisch wrote:

    > You could replace sys.stdout by a class that splits the written text
    > depending on the current thread. It might look roughly like this:
    >
    > class ThreadPrinter:
    > def __init__(self):
    > _.fhs = {}
    >
    > def write(self, value):
    > f = _.fhs.get(threading.currentThread(),
    > open(get_some_nice_file_name(), "w")
    > f.write(value)
    > _.fhs[threading.currentThread()] = f


    Have you run this code? It looks to me suspiciously as
    though it will raise an exception on the second write
    call in any given thread, as the non-shortcircuiting call
    to .get() tries to open the nice_file in write mode for
    a second time.

    Also, what's "_" supposed to be here? self?

    -Peter
    Peter Hansen, Sep 15, 2004
    #4
  5. Peter Hansen wrote:

    > Have you run this code? It looks to me suspiciously as
    > though it will raise an exception on the second write
    > call in any given thread, as the non-shortcircuiting call
    > to .get() tries to open the nice_file in write mode for
    > a second time.


    Nope, didn't run it - and you are right of course. I should have said more
    clearly that the code was untested - I thought that describing it as

    "might look roughly like this"

    would suffice.

    > Also, what's "_" supposed to be here?  self?


    Yup it is - I use _ for self - I tried to adapt to the common standard for
    the post, but failed in the middle of it. Sorry for the confusion.

    --
    Regards,

    Diez B. Roggisch
    Diez B. Roggisch, Sep 15, 2004
    #5
  6. aurora

    Peter Hansen Guest

    Diez B. Roggisch wrote:

    > Peter Hansen wrote:
    >>Have you run this code?

    >
    > Nope, didn't run it - and you are right of course. I should have said more
    > clearly that the code was untested - I thought that describing it as
    >
    > "might look roughly like this"
    >
    > would suffice.


    That probably would have sufficed, but I'm one of those people
    that tends not to read the documentation. I just jumped to
    the code. ;-)

    -Peter
    Peter Hansen, Sep 15, 2004
    #6
  7. aurora

    aurora Guest

    On Wed, 15 Sep 2004 23:14:47 +0200, Diez B. Roggisch <>
    wrote:

    > aurora wrote:
    >
    >> This may sound a little crazy. I capture the output of one class by
    >> redirecting the sys.stdout. However the is another threading running at
    >> the same time and occasionaly it output some messages to the redirected
    >> sys.stdout irreleveant to the output I want to capture. Is there a way
    >> to
    >> redirect output specific to some threads?

    >
    > You could replace sys.stdout by a class that splits the written text
    > depending on the current thread. It might look roughly like this:
    >
    > class ThreadPrinter:
    > def __init__(self):
    > _.fhs = {}
    >
    > def write(self, value):
    > f = _.fhs.get(threading.currentThread(),
    > open(get_some_nice_file_name(), "w")
    > f.write(value)
    > _.fhs[threading.currentThread()] = f
    >
    > Now before starting your threads, replace sys.stdout with an instance of
    > ThreadPrinter:
    >
    > sys.stdout = ThreadPrinter()
    >
    >


    Thanks this is a nice idea. I hope Python would actually support the '_'
    syntax. The self really reduce readablity, especially if you have several
    of them in one line.
    aurora, Sep 15, 2004
    #7
  8. aurora

    Peter Hansen Guest

    aurora wrote:

    > On Wed, 15 Sep 2004 23:14:47 +0200, Diez B. Roggisch
    >> class ThreadPrinter:
    >> def __init__(self):
    >> _.fhs = {}
    >>
    >> def write(self, value):
    >> f = _.fhs.get(threading.currentThread(),

    .....

    > Thanks this is a nice idea. I hope Python would actually support the
    > '_' syntax. The self really reduce readablity, especially if you have
    > several of them in one line.


    It does! One just has to be consistent within each function.
    Diez changed the code from something like this:

    def __init__(_):
    _.fhs = {}

    def write(_, value):
    f = _.fhs.get(threading.currentThread(),
    ....

    Some would argue that this is actually less readable, however,
    since it uses punctuation instead of a word. If nothing else,
    you run into a bit of a conflict between your own technique,
    with "_", and the vast majority of the rest of the Python world,
    which uses "self" exclusively, leading to situations like this
    one...

    (I think if I had a routine that really heavily used self,
    to the obvious detriment of readability, and it wasn't clear
    how else to improve it, I would use a local assignment at
    the top to make a shorter name, perhaps "s", or even "_" --
    but I wouldn't use the possibility of such a thing as a
    justification for using _ everywhere.)

    -Peter
    Peter Hansen, Sep 16, 2004
    #8
  9. aurora

    aurora Guest

    too many self

    Peter Hansen wrote:
    > aurora wrote:
    >
    >> On Wed, 15 Sep 2004 23:14:47 +0200, Diez B. Roggisch
    >>
    >>> class ThreadPrinter:
    >>> def __init__(self):
    >>> _.fhs = {}
    >>>
    >>> def write(self, value):
    >>> f = _.fhs.get(threading.currentThread(),

    >
    > ....
    >
    >> Thanks this is a nice idea. I hope Python would actually support the
    >> '_' syntax. The self really reduce readablity, especially if you have
    >> several of them in one line.

    >
    >
    > It does! One just has to be consistent within each function.
    > Diez changed the code from something like this:
    >
    > def __init__(_):
    > _.fhs = {}
    >
    > def write(_, value):
    > f = _.fhs.get(threading.currentThread(),
    > ...
    >
    > Some would argue that this is actually less readable, however,
    > since it uses punctuation instead of a word. If nothing else,
    > you run into a bit of a conflict between your own technique,
    > with "_", and the vast majority of the rest of the Python world,
    > which uses "self" exclusively, leading to situations like this
    > one...
    >
    > (I think if I had a routine that really heavily used self,
    > to the obvious detriment of readability, and it wasn't clear
    > how else to improve it, I would use a local assignment at
    > the top to make a shorter name, perhaps "s", or even "_" --
    > but I wouldn't use the possibility of such a thing as a
    > justification for using _ everywhere.)
    >
    > -Peter


    Didn't aware that _ itself is a valid identifier! True, you don't really
    want to do things differently from convention. I'm just ranting about
    the verbosity of self.

    This doesn't take a complicated statement to make it really clumsy. Some
    simple statement would look like this:

    if self.max < self.list[self.index]:
    self.max = self.list[self.index]:

    Replacing self with _, depends on one's aesthetic, it could be ugly or
    it could be cleaner. I like it that it is not a word and it does not
    interfere with the keywords that's really relevant.

    if _.max < _.list[_.index]:
    _.max = _.list[_.index]:

    Of couse I think this syntax the best:

    if max < list[index]:
    max = list[index]:

    This remind me of those awful Hungarian notation.

    aurora
    aurora, Sep 16, 2004
    #9
  10. Re: too many self

    aurora <> wrote:
    ...
    > Of couse I think this syntax the best:
    >
    > if max < list[index]:
    > max = list[index]:


    Just to ensure that the best approach,
    self.max = max(self.max, self.list[index])
    isn't available any more, _and_ you can't use the list built-in name any
    more either? What a scoop!


    > This remind me of those awful Hungarian notation.


    Explicit scope denotation, and Hungarian notation (which prefixes names
    with type-connected prefix strings), have essentially nothing to do with
    each other, of course. Making classes implicit scopes (like, say, C++,
    but differently from, say, Modula-3) is simply a horrid mess, where you
    can't use bare names safely without carefully studying all the internals
    of all your ancestor classes... and if any such ancestor ever adds a
    private name it can break every subclass which _did_ use bare names. A
    bad idea even in a language where the compiler can find out statically
    where every name comes from (because human readers can't), just as bad
    as "from foo import *" in Python or "using namespace foo" in C++ except
    that you can't avoid it by just eschewing one misdesigned construct.

    In a language where even the compiler _cannot_ tell statically which
    bare names come from where (except for functions' locals), criticizing
    the language design choice of _not_ making classes into implcit scopes
    doesn't even _verge_ on the ridiculous -- it plunges right deep into it.


    Alex
    Alex Martelli, Sep 16, 2004
    #10
  11. Diez B. Roggisch <> wrote:

    > aurora wrote:
    >
    > > This may sound a little crazy. I capture the output of one class by
    > > redirecting the sys.stdout. However the is another threading running at
    > > the same time and occasionaly it output some messages to the redirected
    > > sys.stdout irreleveant to the output I want to capture. Is there a way to
    > > redirect output specific to some threads?

    >
    > You could replace sys.stdout by a class that splits the written text
    > depending on the current thread. It might look roughly like this:
    >
    > class ThreadPrinter:
    > def __init__(self):
    > _.fhs = {}
    >
    > def write(self, value):
    > f = _.fhs.get(threading.currentThread(),
    > open(get_some_nice_file_name(), "w")
    > f.write(value)
    > _.fhs[threading.currentThread()] = f


    Not a bad general idea, but you need a better implementation of the
    "thread-local storage" design pattern than just a bare dictionary like
    this 'fhs' dict. In Python 2.4, threading.local gives you such an
    implementation. If you need to work in Python 2.3, it's more work, but
    there are cookbook recipes (on Activestate's site) which can help.


    Alex
    Alex Martelli, Sep 16, 2004
    #11
  12. aurora

    aurora Guest

    Re: too many self

    Alex Martelli wrote:
    > aurora <> wrote:
    > ...
    >
    >>Of couse I think this syntax the best:
    >>
    >> if max < list[index]:
    >> max = list[index]:

    >
    >
    > Just to ensure that the best approach,
    > self.max = max(self.max, self.list[index])
    > isn't available any more, _and_ you can't use the list built-in name any
    > more either? What a scoop!
    >
    >
    >
    >>This remind me of those awful Hungarian notation.

    >
    >
    > Explicit scope denotation, and Hungarian notation (which prefixes names
    > with type-connected prefix strings), have essentially nothing to do with
    > each other, of course. Making classes implicit scopes (like, say, C++,
    > but differently from, say, Modula-3) is simply a horrid mess, where you
    > can't use bare names safely without carefully studying all the internals
    > of all your ancestor classes... and if any such ancestor ever adds a
    > private name it can break every subclass which _did_ use bare names. A
    > bad idea even in a language where the compiler can find out statically
    > where every name comes from (because human readers can't), just as bad
    > as "from foo import *" in Python or "using namespace foo" in C++ except
    > that you can't avoid it by just eschewing one misdesigned construct.
    >
    > In a language where even the compiler _cannot_ tell statically which
    > bare names come from where (except for functions' locals), criticizing
    > the language design choice of _not_ making classes into implcit scopes
    > doesn't even _verge_ on the ridiculous -- it plunges right deep into it.
    >
    >
    > Alex


    I'm not making any serious criticism on the language or how it should do
    name binding. I'm ranting about having to attach a prefix to names often
    make simple things look complicated. An annoyance when it has to be done
    very often. You got the point?
    aurora, Sep 17, 2004
    #12
  13. aurora

    Elbert Lev Guest

    Re: too many self

    aurora <> wrote in message news:<>...
    > Peter Hansen wrote:
    > > aurora wrote:
    > >
    > >> On Wed, 15 Sep 2004 23:14:47 +0200, Diez B. Roggisch
    > >>
    > >>> class ThreadPrinter:
    > >>> def __init__(self):
    > >>> _.fhs = {}
    > >>>
    > >>> def write(self, value):
    > >>> f = _.fhs.get(threading.currentThread(),

    > >
    > > ....
    > >
    > >> Thanks this is a nice idea. I hope Python would actually support the
    > >> '_' syntax. The self really reduce readablity, especially if you have
    > >> several of them in one line.

    > >
    > >
    > > It does! One just has to be consistent within each function.
    > > Diez changed the code from something like this:
    > >
    > > def __init__(_):
    > > _.fhs = {}
    > >
    > > def write(_, value):
    > > f = _.fhs.get(threading.currentThread(),
    > > ...
    > >
    > > Some would argue that this is actually less readable, however,
    > > since it uses punctuation instead of a word. If nothing else,
    > > you run into a bit of a conflict between your own technique,
    > > with "_", and the vast majority of the rest of the Python world,
    > > which uses "self" exclusively, leading to situations like this
    > > one...
    > >
    > > (I think if I had a routine that really heavily used self,
    > > to the obvious detriment of readability, and it wasn't clear
    > > how else to improve it, I would use a local assignment at
    > > the top to make a shorter name, perhaps "s", or even "_" --
    > > but I wouldn't use the possibility of such a thing as a
    > > justification for using _ everywhere.)
    > >
    > > -Peter

    >
    > Didn't aware that _ itself is a valid identifier! True, you don't really
    > want to do things differently from convention. I'm just ranting about
    > the verbosity of self.
    >
    > This doesn't take a complicated statement to make it really clumsy. Some
    > simple statement would look like this:
    >
    > if self.max < self.list[self.index]:
    > self.max = self.list[self.index]:
    >
    > Replacing self with _, depends on one's aesthetic, it could be ugly or
    > it could be cleaner. I like it that it is not a word and it does not
    > interfere with the keywords that's really relevant.
    >
    > if _.max < _.list[_.index]:
    > _.max = _.list[_.index]:
    >
    > Of couse I think this syntax the best:
    >
    > if max < list[index]:
    > max = list[index]:
    >
    > This remind me of those awful Hungarian notation.
    >
    > aurora



    Please do not do this, please...
    Recently I was working with a modules written by a programmer,
    who used "_" instead of "self".
    This was so "not in line" with the rest of the system,
    that it took me extra hour or two to get accustomed
    and switch back and force (self in "standard" modules and _ in "economical").
    Elbert Lev, Sep 17, 2004
    #13
  14. aurora

    Ville Vainio Guest

    Re: too many self

    >>>>> "aurora" == aurora <> writes:


    aurora> I'm not making any serious criticism on the language or
    aurora> how it should do name binding. I'm ranting about having to
    aurora> attach a prefix to names often make simple things look
    aurora> complicated. An annoyance when it has to be done very
    aurora> often. You got the point?

    Why don't you write a preprocessor that converts every .foo to
    self.foo?

    --
    Ville Vainio http://tinyurl.com/2prnb
    Ville Vainio, Sep 17, 2004
    #14
  15. Re: too many self

    aurora <> wrote:
    ...
    > I'm not making any serious criticism on the language or how it should do
    > name binding. I'm ranting about having to attach a prefix to names often
    > make simple things look complicated. An annoyance when it has to be done
    > very often. You got the point?


    No. If you had to "attach a prefix", it might perhaps be a problem.
    But, in Python, you don't have to do any such thing. You use bare names
    for locals, globals, and built-ins, and compound names for attributes of
    objects. How can it "make simple things look complicated" to use a
    compound name for an object attribute? It indicates exactly what's
    going on: you're accessing or rebinding an attribute -- no more, no
    less. Just like, e.g., x is how you refer to an item of x, so is
    x.foo how you refer to an attribute of x.

    If what you mean to say (as opposed to what you actually wrote) is to
    support some kind of 'with' or 'using' statement, such as:

    with glek[idx]:
    .total += .partial
    .partial = current_time
    .laps += 1

    so that several accesses to, and rebindings of, attributes of the same
    object, can be compactly and conveniently indicated, that, of course, is
    quite a different issue, which has often been discussed in the past,
    both here and on python-dev. Yet there isn't even a PEP for it, yet, I
    believe -- a bad sign: it suggests nobody is keen enough on it to
    summarize the case for and against and nail down the specs. I guess the
    problem is that if you see this as equivalent to, say:

    _ = glek[idx]:
    _.total += _.partial
    _.partial = current_time
    _.laps += 1

    it's not all that clear that the with notation is buying you all that
    much -- visually distinguishing that current_time is not an attribute
    name is about as hard in both cases, for example, maybe a bit harder if
    the hypothetical new construct were in use... and without said
    hypothetical new construct you have a better chance to make your code
    clearer by using a visually distinguished name rather than a notation
    which appears to try to HIDE the crucial issue of which name are bare
    ones, and which names are compound!

    Richer semantics for 'with'/'using' are of course possible, but harder
    to pin down in detail, and even more controversial. Still, until
    somebody DOES care enough, for or against, to come up with a PEP, rather
    than rants on this NG, nothing much is going to happen re this idea.


    Alex
    Alex Martelli, Sep 17, 2004
    #15
  16. Re: too many self

    On Fri, 17 Sep 2004 08:54:11 +0200, (Alex Martelli) wrote:

    >aurora <> wrote:
    > ...
    >> I'm not making any serious criticism on the language or how it should do
    >> name binding. I'm ranting about having to attach a prefix to names often
    >> make simple things look complicated. An annoyance when it has to be done
    >> very often. You got the point?

    >
    >No. If you had to "attach a prefix", it might perhaps be a problem.
    >But, in Python, you don't have to do any such thing. You use bare names
    >for locals, globals, and built-ins, and compound names for attributes of
    >objects. How can it "make simple things look complicated" to use a
    >compound name for an object attribute? It indicates exactly what's
    >going on: you're accessing or rebinding an attribute -- no more, no
    >less. Just like, e.g., x is how you refer to an item of x, so is
    >x.foo how you refer to an attribute of x.
    >
    >If what you mean to say (as opposed to what you actually wrote) is to
    >support some kind of 'with' or 'using' statement, such as:
    >
    >with glek[idx]:
    > .total += .partial
    > .partial = current_time
    > .laps += 1
    >
    >so that several accesses to, and rebindings of, attributes of the same
    >object, can be compactly and conveniently indicated, that, of course, is
    >quite a different issue, which has often been discussed in the past,
    >both here and on python-dev. Yet there isn't even a PEP for it, yet, I
    >believe -- a bad sign: it suggests nobody is keen enough on it to
    >summarize the case for and against and nail down the specs. I guess the
    >problem is that if you see this as equivalent to, say:
    >
    >_ = glek[idx]:
    >_.total += _.partial
    >_.partial = current_time
    >_.laps += 1
    >
    >it's not all that clear that the with notation is buying you all that
    >much -- visually distinguishing that current_time is not an attribute
    >name is about as hard in both cases, for example, maybe a bit harder if
    >the hypothetical new construct were in use... and without said
    >hypothetical new construct you have a better chance to make your code
    >clearer by using a visually distinguished name rather than a notation
    >which appears to try to HIDE the crucial issue of which name are bare
    >ones, and which names are compound!
    >
    >Richer semantics for 'with'/'using' are of course possible, but harder
    >to pin down in detail, and even more controversial. Still, until
    >somebody DOES care enough, for or against, to come up with a PEP, rather
    >than rants on this NG, nothing much is going to happen re this idea.
    >

    It occurs to me that "with obj_expr:" selects a specific name space and
    access mechanism (attribute access, which looks through base classes for
    decriptors etc), whereas one could expand the concept of what the leading
    dot means as an access operation. E.g., the normal default spelled out might be

    with(glek[idx], dotop=with.attr): # with.attr would be a get/set pair for attributes
    .total += .partial
    ...

    Or you could limit yourself to an instance dict by alternate meaning
    for '.' access, e.g.,

    with(vars(inst), dotop=with.item): # with.item would be get/set for items
    .total += .partial # meaning v=vars(inst); v['total'] += v['partial']; del v
    ...

    or you could specify a list of spaces to chase in explicit order, e.g.,
    object attribute spaces:

    with(a,b,c):
    x = .xxx # a.xxx if avail, else b.xxx, else c.xxx, else AttributeError

    Anyway, you get the general idea. A way to spell out access to any alternative
    name space or search path through several. ;-)

    Regards,
    Bengt Richter
    Bengt Richter, Sep 17, 2004
    #16
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Roman Neuhauser
    Replies:
    0
    Views:
    653
    Roman Neuhauser
    Apr 4, 2005
  2. Replies:
    2
    Views:
    648
    velle
    Jan 5, 2006
  3. Replies:
    0
    Views:
    464
  4. Mitchell L Model

    sys.stdout vs. sys.stderr

    Mitchell L Model, Jan 11, 2010, in forum: Python
    Replies:
    2
    Views:
    564
    Nobody
    Jan 11, 2010
  5. Michel Albert
    Replies:
    5
    Views:
    1,383
    Daniel Dehennin
    Oct 24, 2012
Loading...

Share This Page