7

I'm working on a connection-like object which implements a context manager. Writing something like this is strongly encouraged:

with MyConnection() as con: # do stuff

Of course one can do this as well:

con = MyConnection() # do stuff con.close()

But failing to close the connection is rather problematic. So closing in the __del__() seems like a good idea:

def __del__(self): self.close()

This looks quite nice, but sometimes leads to errors:

Exception ignored in: [...] Traceback (most recent call last): File "...", line xxx, in __del__() TypeError: 'NoneType' object is not callable

It appears as if sometimes the close method is already destroyed, when __del__() is called.

So I'm looking for a nice way to encourage python to close the connection properly on destruction. If possible I would like to avoid code duplication in close() and __del__()

Improve this question
5
  • Possible duplicate: stackoverflow.com/questions/865115/… Commented Jul 7, 2014 at 13:20
  • If you're writing a context manager, use __exit__ Commented Jul 7, 2014 at 13:21
  • thanks for your comments - I know that using the context manager is a nice solution - yet users can't be forced to use it. I'd like to avoid system crashes if users fail to close connections properly. Commented Jul 7, 2014 at 13:26
  • To avoid code duplication, have close call __del__. And don't get too wrapped up in protecting your users -- they are, after all, programmers and should be smart enough to use the API you have given them -- just make sure you have good docs, and the rest is on them. Commented Jul 7, 2014 at 15:58
  • Calling __del__ in the close method is somewhat ugly as well. I think you are right - It's probably not a good idea to focus too much on users misusing the code. Especially when the code changes are rather hackish. Commented Jul 7, 2014 at 18:12

6 Answers 6

Reset to default
13

If you really want to prevent the user from not closing the connection, you could just init it only in __enter__ or you may add a flag spotting the fact it has not been initialized by a context manager. For instance, something like

class MyConnection(object): safely_initialized = False def __enter__(self): # Init your connection self.safely_initialized = True return self def do_something(self): if not self.safely_initialized: raise Exception('You must initialize the connection with a context manager!') # Do something def __exit__(self, type, value, traceback):   # Close your connection

That way the connection won't be initialized unless within a context manager.

Improve this answer
Sign up to request clarification or add additional context in comments.

5 Comments

So instead of trying to clean up after a poor choice on their part, don't even run unless they use the supported method. Nice.
Yes, that would probably work. Yet it would force users to use the with statement. I'm not sure if I want to do that.
I think this is the best solution so far. Yet I'm not sure if I like it better than trying to nuke connections in the __del__ method. This would lead to some lines of ugly code duplication, but does not force the user to use the with statement. It's not very important for me that the connections are closed properly - just having too many is not a good idea.
As the others mentionned, __del__ should be avoided because it can lead to unpredictable results. So if you don't want to force the users, then you'll have to accept some not-well closed connections (or have a look at @ethan-furman answer).
Yes, it's probably not a good idea to use the __del__ method for connection cleanups. I'll wait until tomorrow - if there are no more ideas I will mark your answer as the correct one.
4

There is no guarantee when __del__ will actually run. Since you are using a with statement, use the __exit__ method instead. __exit__ will be called as soon as the with statement is finished, no matter how the statement completes (normally, with an exception, etc).

Improve this answer

1 Comment

yes my current implementation uses a context manager and users are encouraged to use it. Yet I'm looking for a cleanup if users fail to use the context manager properly.
2

It is true you cannot force your users to use good programming techniques, and if they refuse to do so you cannot be responsible for them.

There is no guarantee of when __del__ will be called -- in some Pythons it is immediate, in others it may not happen until interpreter shutdown. So, while not a very good option, using atexit may be viable -- just be sure that the function you register is smart enough to check if the resource has already been closed/destroyed.

Improve this answer

1 Comment

Yes, it's true that I'm not responsible for bad programming - yet it's nice to avoid crashes and problems if possible. While it's true that I can't be sure that all connections are killed - most of them probably will. The rest will get nuked during interpreter shutdown.
1

You could try calling close in __del__, and ignore any exceptions:

del __del__(self): try: self.close() except TypeError: pass
Improve this answer

2 Comments

yes, I thought about that, but I think this does not tackle the key problem. For me, calling the close method in del seems like a bad idea in general, since it is not guaranteed that this method even exits anymore.
@cel: If the interpreter is shutting down, you have no guarentees about anything.
1

weakref.finalize() lets you perform an action when an object is garbage collected, or the program exits. It's available from Python 3.4 onwards.

When you create your object you can make a call to finalize(), providing it a callback that cleans up the resources your object holds.

The Python docs provide several examples of its use:

>>> import weakref >>> class Object: ... pass ... >>> kenny = Object() >>> weakref.finalize(kenny, print, "You killed Kenny!") <finalize object at ...; for 'Object' at ...> >>> del kenny You killed Kenny!

And this example of a class representing a temporary directory whose contents are deleted when:

  • its remove() method is called
  • it gets garbage collected
  • program exits

Whichever happens first.

class TempDir: def __init__(self): self.name = tempfile.mkdtemp() self._finalizer = weakref.finalize(self, shutil.rmtree, self.name) def remove(self): self._finalizer() @property def removed(self): return not self._finalizer.alive
Improve this answer

Comments

0

You may define a close() method and call it on is_open condition both in __exit__ and __del__ methods as follows:

class MyContext: def __init__(self, *args): self.is_open = False self.args = args self.open(*self.args) def __enter__(self): return self def __del__(self): self.close() def __exit__(self, *args): self.close() def open(self, *args): if not self.is_open: self.is_open = True print("opening: ", args) return self def close(self): if self.is_open: self.is_open = False print("closing: ", self.args)

Here is a usage example WITHOUT a context manager:

def init_way(): c = MyContext("path", "openparam") init_way()

Possible output:

opening: ('path', 'openparam') closing: ('path', 'openparam')

And another example: using the same class as a context manager this time:

def context_way(): with MyContext("path", "openparam") as c: print("in with ...") context_way()

Possible output:

opening: ('path', 'openparam') in with ... closing: ('path', 'openparam')
Improve this answer

Comments

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.