nesting context managers

[Ulrich Eckhardt]

Hi!

Let us assume I had a class HTTPClient that has a socket for HTTP and a
logfile for filing logs. I want to make this HTTPClient a context
manager, so that I can write

with HTTPClient(url) as client:
pass

and reliably have both the socket and the logfile closed. The easy way
is wrong

def __enter__(self):
with self._mysock, self._myfile:
return self

because it closes the file and socket as part of the return statement.
Of course I can go the hard way, code everything myself, but I wonder if
there isn't some useful tool that would help me achieve this (common?) task.

Thanks!

Uli

 

[Rami Chowdhury]

Hi!

Let us assume I had a class HTTPClient that has a socket for HTTP and a
logfile for filing logs. I want to make this HTTPClient a context manager,
so that I can write

 with HTTPClient(url) as client:
     pass

and reliably have both the socket and the logfile closed. The easy way is
wrong

 def __enter__(self):
     with self._mysock, self._myfile:
         return self

It seems like some of the functions in the contextlib module might
help? You could try and reorganize your code so that you can use the
@contextmanager decorator, for instance?

That having been said, it doesn't seem that difficult to me to code
your own simple __exit__ method if you're already coding up __enter__
?

HTH,
Rami

 

[Ulrich Eckhardt]

Am 20.12.2011 15:15, schrieb Ulrich Eckhardt:

Let us assume I had a class HTTPClient that has a socket for HTTP and a
logfile for filing logs. I want to make this HTTPClient a context
manager, so that I can write

with HTTPClient(url) as client:
pass

Actually, I overestimated the task:

class HTTPClient(url):
def __enter__(self):
return self
def __exit__(self, ctx_type, ctx_val, ctx_tb):
self.close()
def close(self):
self._myfile.close()
self._mysocket.close()

I'll simply ignore the fact that closing a file can fail if you can't
flush buffers. Now, my error was that I have to somehow invoke the
file's and socket's enter function in my client's enter function. The
default for all files and sockets is that they are already open, they
are not opened in the enter function! That way, the client remains
usable outside a with context but gives me the desired guarantees inside
one.

For the case of on-demand allocated stuff, I could write the enter
function like this:

def __enter__(self):
# allocate resources
f = ... # open file
s = ... # connect socket
# attach after completion
self._myfile = f
self._mysocket = s

To be extra safe or in more complicated scenarios, I could wrap this in
a try-except and explicitly close those that were already created, but
normally I'd expect the garbage collector to do that for me ... or am I
then implicitly assuming a specific implementation?


Uli

 

[Rami Chowdhury]

To be extra safe or in more complicated scenarios, I could wrap this in a
try-except and explicitly close those that were already created, but
normally I'd expect the garbage collector to do that for me ... or am I then
implicitly assuming a specific implementation?

I'm no expert but I believe the basic garbage collection behavior is
part of the language, and it's only details of breaking cycles that
are specific to CPython -- can anyone correct me if I'm wrong?

 

[Ethan Furman]

I'm no expert but I believe the basic garbage collection behavior is
part of the language, and it's only details of breaking cycles that
are specific to CPython -- can anyone correct me if I'm wrong?

Garbage collection is part of the language, but how, and when, is
implementation specific.

Resource management (beyond basic memory allocation) should be handled
directly. Python 3 encourages this by complaining if there were still
open files when it shuts down.

~Ethan~

 

[Ian Kelly]

Am 20.12.2011 15:15, schrieb Ulrich Eckhardt:



Actually, I overestimated the task:

 class HTTPClient(url):
     def __enter__(self):
         return self
     def __exit__(self, ctx_type, ctx_val, ctx_tb):
         self.close()
     def close(self):
         self._myfile.close()
         self._mysocket.close()

I'll simply ignore the fact that closing a file can fail if you can't flush
buffers. Now, my error was that I have to somehow invoke the file's and
socket's enter function in my client's enter function. The default for all
files and sockets is that they are already open, they are not opened in the
enter function! That way, the client remains usable outside a with context
but gives me the desired guarantees inside one.

For the case of on-demand allocated stuff, I could write the enter function
like this:

 def __enter__(self):
     # allocate resources
     f = ... # open file
     s = ... # connect socket
     # attach after completion
     self._myfile = f
     self._mysocket = s

To be extra safe or in more complicated scenarios, I could wrap this in a
try-except and explicitly close those that were already created, but
normally I'd expect the garbage collector to do that for me ... or am I then
implicitly assuming a specific implementation?

For full generality, copy the code for the contextlib.nested function
in Python 2.7 or 3.1 and use that. Don't use contextlib.nested
itself, though, because it's deprecated and does not exist in Python
3.2.

Cheers,
Ian

 

[Ian Kelly]

For full generality, copy the code for the contextlib.nested function
in Python 2.7 or 3.1 and use that.  Don't use contextlib.nested
itself, though, because it's deprecated and does not exist in Python
3.2.

Also note that in versions that do support the nested with syntax, you
can do something like:

class HTTPClient(url):
@contextlib.contextmanager
def _contextmanager(file, socket):
with file, socket:
yield
def __enter__(self):
self._cm = self._contextmanager(self._myfile, self._mysocket)
return self._cm.__enter__()
def __exit__(self, exc_type, exc_value, traceback):
try:
return self._cm.__exit__(exc_type, exc_value, traceback)
finally:
del self._cm

Cheers,
Ian