I’ve always been intrigued by the capabilities of dynamic languages. These include features like adding attributes to objects on-the-fly, modifying an object’s class, altering inheritance chains, and intercepting attribute access. Python offers a rich set of these features, including the powerful concept of metaclasses. However, there are some nuances, rules, and limitations to these features in Python. Let’s explore some that I’ve encountered.

A hallmark of dynamic languages is the ability to add new attributes to an object directly. In Python, we can easily do this with custom classes and their instances. However, attempting to add an attribute to an instance of the object class directly results in an “‘object’ object has no attribute” Exception. This limitation stems from instances of the object class lacking a __dict__ attribute, which is where instance attributes are stored (unless using slots). Creating a custom class, even an empty one inheriting from object, provides instances with a __dict__, allowing attribute assignment. More details on this can be found here.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

o1 = object()
print(dir(o1))
# o1 does not have __dict__
# ['__class__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__']

try:
    o1.city = "Paris"
except Exception as ex:
    print(f"Exception: {ex}")
#Exception: 'object' object has no attribute 'city'

class Ob(object):
    pass
    
o2 = Ob()
print(dir(o2))
# o2 has __dict__
#['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__']

o2.city = "Paris"
print(o2.city)
#Paris 

In Python, every object has a __class__ attribute referencing its class (e.g., ‘Class1’). This attribute can be dynamically reassigned to a different class (e.g., ‘Class2’), effectively changing the object’s behavior to align with the new class. Classes, in turn, have a __bases__ attribute, a tuple representing their inheritance chain. Modifying this tuple seems like a way to dynamically alter inheritance. However, there’s a catch. Attempting to reassign __bases__ for a class directly inheriting from object throws an exception: __bases__ assignment: ‘XX’ deallocator differs from ‘object’. Interestingly, if the class inherits from another class (even an empty one derived from object), the reassignment works without issue. The exception seems tied to the garbage collection mechanism a bug that is known and remains open since 2003, which is peculiar. When successful, reassigning __bases__ triggers a recalculation of the class’s MRO (method resolution order), reflected in the __mro__ attribute.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

class A1:
    pass

class A2:
    pass

class B:
    pass

# for classes that directly inherit from object __bases__ reassignment fails
try:
    B.__bases__ = (A1, A2)
except Exception as ex:
    print(f"exception! {ex}")
# exception! __bases__ assignment: 'A1' deallocator differs from 'object'

# but if we inherit from any other class (for example this "empty" one) it works
class Empty:
    pass

class C (Empty):
    pass

print(f"C.__mro__ before __bases__ reassignment: {C.__mro__}")
# C.__mro__ before __bases__ reassignment: (, , )
C.__bases__ = (A1, A2)
print(f"C.__mro__ before __bases__ reassignment: {C.__mro__}")
# C.__mro__ after __bases__ reassignment: (, , , ) 

Classes themselves do not possess a __metaclass__ attribute. This makes sense as the metaclass dictates a class’s creation; it’s the class object’s __class__ attribute that references the metaclass. One might assume that reassigning a class’s __class__ attribute could change its metaclass. However, this only works for instances, not for classes. Attempting this on a class throws an exception: __class__ assignment only supported for mutable types or ModuleType subclasses.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

class Meta1(type):
    pass

class A2(metaclass=Meta1):
    def __init__(self,  name):
        self.name = name

print(f"A2 metaclass: {A2.__class__}")
# A2 metaclass: print(f"A2 metaclass: {type(A2)}")
# A2 metaclass: try:
    A2.__class__ = type
except BaseException as ex:
    print(f"Exception! trying to change the metaclass: {ex}")
# Exception! trying to change the metaclass: __class__ assignment only supported for mutable types or ModuleType subclasses 

While metaclasses primarily govern class creation, changing a class’s metaclass after the fact has no effect. However, they can influence instance creation by providing a custom __call__ method within the metaclass. Instantiating a class (e.g., a = A()) invokes __call__ on the class’s metaclass. By overriding __call__ in the metaclass, we can customize instance creation.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

def _custom_call(cls, *args, **kwargs):
    print(f"invoking {cls.__name__} construction")
    instance = type.__call__(cls, *args, **kwargs)
    # Additional customization if needed
    return instance

Meta1.__call__ = _custom_call
a2 = A2("Iyan")
# invoking A2 construction
print(f"a2 type: {type(a2).__name__}")
# a2 type: A2
print(f"a2.name: {a2.name}")
# a2.name: Iyan 

Metaclasses have another less-known use: defining a custom MRO for classes. I came across this feature in this post. By overriding the mro() method in the metaclass, we can specify a custom __mro__ for classes of that metaclass, which Python will use instead of the default type.mro() implementation. Note that you cannot directly reassign a class’s __mro__ attribute to change its MRO, as it is read-only. There’s an involved discussion about a potential workaround for this here.