Python: changing methods and attributes at runtime

This example shows the differences between adding a method to a class and to an instance.

>>> class Dog(): ... def __init__(self, name): ... self.name = name ... >>> skip = Dog('Skip') >>> spot = Dog('Spot') >>> def talk(self): ... print 'Hi, my name is ' + self.name ... >>> Dog.talk = talk # add method to class >>> skip.talk() Hi, my name is Skip >>> spot.talk() Hi, my name is Spot >>> del Dog.talk # remove method from class >>> skip.talk() # won't work anymore Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: Dog instance has no attribute 'talk' >>> import types >>> f = types.MethodType(talk, skip, Dog) >>> skip.talk = f # add method to specific instance >>> skip.talk() Hi, my name is Skip >>> spot.talk() # won't work, since we only modified skip Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: Dog instance has no attribute 'talk' 

A possibly interesting alternative to using types.MethodType in:

>>> f = types.MethodType(talk, puppy, Dog) >>> puppy.talk = f # add method to specific instance 

would be to exploit the fact that functions are descriptors:

>>> puppy.talk = talk.__get__(puppy, Dog) 

I wish to create a class in Python that I can add and remove attributes and methods. How can I acomplish that?

You can add and remove attributes and methods to any class, and they'll be available to all instances of the class:

>>> def method1(self): pass >>> def method1(self): print "method1" >>> def method2(self): print "method2" >>> class C(): pass >>> c = C() >>> c.method() Traceback (most recent call last): File "<pyshell#62>", line 1, in <module> c.method() AttributeError: C instance has no attribute 'method' >>> C.method = method1 >>> c.method() method1 >>> C.method = method2 >>> c.method() method2 >>> del C.method >>> c.method() Traceback (most recent call last): File "<pyshell#68>", line 1, in <module> c.method() AttributeError: C instance has no attribute 'method' >>> C.attribute = "foo" >>> c.attribute 'foo' >>> c.attribute = "bar" >>> c.attribute 'bar' 

you can just assign directly to the class (either by accessing the original class name or via __class__ ):

class a : pass ob=a() ob.__class__.blah=lambda self,k: (3, self,k) ob.blah(5) ob2=a() ob2.blah(7) 

will print

(3, <__main__.a instance at 0x7f18e3c345f0>, 5) (3, <__main__.a instance at 0x7f18e3c344d0>, 7) 

Simply:

f1 = lambda:0 #method for instances f2 = lambda _:0 #method for class class C: pass #class c1,c2 = C(),C() #instances print dir(c1),dir(c2) #add to the Instances c1.func = f1 c1.any = 1.23 print dir(c1),dir(c2) print c1.func(),c1.any del c1.func,c1.any #add to the Class C.func = f2 C.any = 1.23 print dir(c1),dir(c2) print c1.func(),c1.any print c2.func(),c2.any 

which results in:

['__doc__', '__module__'] ['__doc__', '__module__'] ['__doc__', '__module__', 'any', 'func'] ['__doc__', '__module__'] 0 1.23 ['__doc__', '__module__', 'any', 'func'] ['__doc__', '__module__', 'any', 'func'] 0 1.23 0 1.23 

another alternative, if you need to replace the class wholesale is to modify the class attribute:

>>> class A(object): ... def foo(self): ... print 'A' ... >>> class B(object): ... def foo(self): ... print 'Bar' ... >>> a = A() >>> a.foo() A >>> a.__class__ = B >>> a.foo() Bar 

Does the class itself necessarily need to be modified? Or is the goal simply to replace what object.method() does at a particular point during runtime?

I ask because I sidestep the problem of actually modifying the class to monkey patch specific method calls in my framework with getattribute and a Runtime Decorator on my Base inheritance object.

Methods retrieved by a Base object in getattribute are wrapped in a Runtime_Decorator that parses the method calls keyword arguments for decorators/monkey patches to apply.

This enables you to utilize the syntax object.method(monkey_patch="mypatch"), object.method(decorator="mydecorator"), and even object.method(decorators=my_decorator_list).

This works for any individual method call (I leave out magic methods), does so without actually modifying any class/instance attributes, can utilize arbitrary, even foreign methods to patch, and will work transparently on sublcasses that inherit from Base (provided they don't override getattribute of course).

import trace def monkey_patched(self, *args, **kwargs): print self, "Tried to call a method, but it was monkey patched instead" return "and now for something completely different" class Base(object): def __init__(self): super(Base, self).__init__() def testmethod(self): print "%s test method" % self def __getattribute__(self, attribute): value = super(Base, self).__getattribute__(attribute) if "__" not in attribute and callable(value): value = Runtime_Decorator(value) return value class Runtime_Decorator(object): def __init__(self, function): self.function = function def __call__(self, *args, **kwargs): if kwargs.has_key("monkey_patch"): module_name, patch_name = self._resolve_string(kwargs.pop("monkey_patch")) module = self._get_module(module_name) monkey_patch = getattr(module, patch_name) return monkey_patch(self.function.im_self, *args, **kwargs) if kwargs.has_key('decorator'): decorator_type = str(kwargs['decorator']) module_name, decorator_name = self._resolve_string(decorator_type) decorator = self._get_decorator(decorator_name, module_name) wrapped_function = decorator(self.function) del kwargs['decorator'] return wrapped_function(*args, **kwargs) elif kwargs.has_key('decorators'): decorators = [] for item in kwargs['decorators']: module_name, decorator_name = self._resolve_string(item) decorator = self._get_decorator(decorator_name, module_name) decorators.append(decorator) wrapped_function = self.function for item in reversed(decorators): wrapped_function = item(wrapped_function) del kwargs['decorators'] return wrapped_function(*args, **kwargs) else: return self.function(*args, **kwargs) def _resolve_string(self, string): try: # attempt to split the string into a module and attribute module_name, decorator_name = string.split(".") except ValueError: # there was no ".", it's just a single attribute module_name = "__main__" decorator_name = string finally: return module_name, decorator_name def _get_module(self, module_name): try: # attempt to load the module if it exists already module = modules[module_name] except KeyError: # import it if it doesn't module = __import__(module_name) finally: return module def _get_decorator(self, decorator_name, module_name): module = self._get_module(module_name) try: # attempt to procure the decorator class decorator_wrap = getattr(module, decorator_name) except AttributeError: # decorator not found in module print("failed to locate decorators %s for function %s." %\ (kwargs["decorator"], self.function)) else: return decorator_wrap # instantiate the class with self.function class Tracer(object): def __init__(self, function): self.function = function def __call__(self, *args, **kwargs): tracer = trace.Trace(trace=1) tracer.runfunc(self.function, *args, **kwargs) b = Base() b.testmethod(monkey_patch="monkey_patched") b.testmethod(decorator="Tracer") #b.testmethod(monkey_patch="external_module.my_patch") 

The downside to this approach is getattribute hooks all access to attributes, so the checking of and potential wrapping of methods occurs even for attributes that are not methods + won't be utilizing the feature for the particular call in question. And using getattribute at all is inherently somewhat complicated.

The actual impact of this overhead in my experience/for my purposes has been negligible, and my machine runs a dual core Celeron. The previous implementation I used introspected methods upon object init and bound the Runtime_Decorator to methods then. Doing things that way eliminated the need to utilize getattribute and reduced the overhead mentioned previously... however, it also breaks pickle (maybe not dill) and is less dynamic then this approach.

The only use cases I have actually come across "in the wild" with this technique were with timing and tracing decorators. However, the possibilities it opens up are extremely wide ranging.

If you have a preexisting class that cannot be made to inherit from a different base (or utilize the technique it's own class definition or in it's base class'), then the whole thing simply does not apply to your issue at all unfortunately.

I don't think setting/removing non-callable attributes on a class at runtime is necessarily so challenging? unless you want classes that inherit from the modified class to automatically reflect the changes in themselves as well... That'd be a whole 'nother can o' worms by the sound of it though.