Inheritance of metaclass

Question

In this well known answer that explains metaclass in Python. It mentions that the __metaclass__ attribute will not be inherited.

But as a matter of fact, I tried in Python:

class Meta1(type):
    def __new__(cls, clsname, bases, dct):
        print "Using Meta1"
        return type.__new__(cls, clsname, bases, dct)

# "Using Meta1" printed
class Foo1:
    __metaclass__ = Meta1

# "Using Meta1" printed
class Bar1(Foo1):
    pass

As expected, both Foo and Bar use Meta1 as metaclass and print string as expected.

But in the following sample, when type(...) is returned instead of type.__new__(...), the metaclass is no longer inherited:

class Meta2(type):
    def __new__(cls, clsname, bases, dct):
        print "Using Meta2"
        return type(clsname, bases, dct)

# "Using Meta2" printed
class Foo2:
    __metaclass__ = Meta2

# Nothing printed
class Bar2(Foo2):
    pass

Inspecting the __metaclass__ and __class__ attributes, I can see:

print Foo1.__metaclass__ # <class '__main__.Meta1'>
print Bar1.__metaclass__ # <class '__main__.Meta1'>
print Foo2.__metaclass__ # <class '__main__.Meta2'>
print Bar2.__metaclass__ # <class '__main__.Meta2'>

print Foo1.__class__ # <class '__main__.Meta1'>
print Bar1.__class__ # <class '__main__.Meta1'>
print Foo2.__class__ # <type 'type'>
print Bar2.__class__ # <type 'type'>

In conclusion:

Both __metaclass__ and __class__ will be inherited from base class.
The creation behavior defined by Meta2 will be used for Foo2, although Foo2.__class__ is actually type.
The __metaclass__ attribute in Bar2 is Meta2, but the creation behavior of Bar2 is not affected. In another word, Bar2 uses type as its "real" metaclass instead of Meta2.

These observations make the inheritance mechanism of __metaclass__ kind of vague to me.

My guess is that:

When directly assigning a class (e.g. Meta1) to the __metaclass__ attribute of another class 'Foo1', It's the __metaclass__ attribute taking effect.
When subclass does not explicitly set __metaclass__ attribute when defining. The __class__ attribute instead of __metaclass__ attribute of base class will decide the "real" metaclass of subclass.

Is my guess correct? How does Python deal with the inheritance of metaclass?

I think it has to do with type.__new__ and type http://stackoverflow.com/questions/2608708/what-is-the-difference-between-type-and-type-new-in-python — Vikas Madhusudana, May 24 '16 at 10:48

score 21 · Accepted Answer · edited Feb 27 '20 at 00:56

You are speculating a lot, while Python's minimalist and "Special cases aren't special enough to break the rules." directive, make it easier to understand than that.

In Python2, a __metaclass__ attribute in the class body is used at class creation time to call the "class" that class will be. Ordinarily it is the class named type. To clarify, that moment is after the parser had parsed the class body, after the compiler had compiled it to a code object, and after it was actually run at program run time, and only if __metaclass__ is explicitly provided in that class body.

So that let's check way goes in a case like:

class A(object):
    __metaclass__ = MetaA

class B(A):
    pass

A has __metaclass__ in its body - MetaA is called instead of type to make it into "class object". B does not have __metaclass__ in its body. After it is created, if you just try to access the __metaclass__ attribute, it is an attribute as anyother, that will be visible because Python willget it from the superclass A. If you check A.__dict__ you will see the __metaclass__ and if you check B.__dict__ don't.

This A.__metaclass__ attribute is not used at all when B is created. If you change it in A before declaring B will still use the same metaclass as A - because Python does use the type of the parent class as metaclass in the absense of the declaration of an explicit __metaclass__.

To illustrate:

In [1]: class M(type): pass

In [2]: class A(object): __metaclass__ = M

In [3]: print "class: {}, metaclass_attr: {}, metaclass_in_dict: {}, type: {}".format(A.__class__, A.__metaclass__, A.__dict__.get("__metaclass__"), type(A))
class: <class '__main__.M'>, metaclass_attr: <class '__main__.M'>, metaclass_in_dict: <class '__main__.M'>, type: <class '__main__.M'>

In [4]: class B(A): pass

In [5]: print "class: {}, metaclass_attr: {}, metaclass_in_dict: {}, type: {}".format(B.__class__, B.__metaclass__, B.__dict__.get("__metaclass__"), type(B))
class: <class '__main__.M'>, metaclass_attr: <class '__main__.M'>, metaclass_in_dict: None, type: <class '__main__.M'>

In [6]: A.__metaclass__ = type

In [8]: class C(A): pass

In [9]: print "class: {}, metaclass_attr: {}, metaclass_in_dict: {}, type: {}".format(C.__class__, C.__metaclass__, C.__dict__.get("__metaclass__"), type(C))
class: <class '__main__.M'>, metaclass_attr: <type 'type'>, metaclass_in_dict: None, type: <class '__main__.M'>

Furthermore, if you try to just create a class through a call to type instead of using a body with a class statement, __metaclass__ is also just an ordinary attribute:

In [11]: D = type("D", (object,), {"__metaclass__": M})

In [12]: type(D)
type

Summing up thus far: The __metaclass__ attribute in Python 2 is only special if it is explicitly placed in the class body declaration, as part of the execution of the class block statement. It is an ordinary attribute with no special properties afterwards.

Python3 both got rid of this strange "__metaclass__ attribute is no good now", and allowed for further customization of the class body by changing the syntax to specify metaclasses. (It is like declared as if it were a "metaclass named parameter" on the class statement itself)

Now, to the second part of what raised your doubts: if in the __new__ method of the metaclass you call type instead of type.__new__, there is no way Python can "know" type is being called from a derived metaclass. When you call type.__new__, you pass as its first parameter the cls attribute your metaclass's __new__ itself was passed by the runtime: that is what marks the resulting class as being an instance of a subclass of type. That is just like inheritance works for any other class in Python - so "no special behaviors" here:

So, spot the difference:

class M1(type):
    def __new__(metacls, name, bases, attrs):
         cls = type.__new__(metacls, name, bases, attrs)
         # cls now is an instance of "M1"
         ...
         return cls


class M2(type):
    def __new__(metacls, name, bases, attrs):
         cls = type(name, bases, attrs)
         # Type does not "know" it was called from within "M2"
         # cls is an ordinary instance of "type"
         ...
         return cls

It can be seen in the interactive prompt:

In [13]: class M2(type):
   ....:     def __new__(metacls, name, bases, attrs):
   ....:         return type(name, bases, attrs)
   ....:     

In [14]: class A(M2): pass

In [15]: type(A)
Out[15]: type

In [16]: class A(M2): __metaclass__ = M2

In [17]: A.__class__, A.__metaclass__
Out[17]: (type, __main__.M2)

(Note that the metaclass __new__ method first parameter is the metaclass itself, therefore more properly named metacls than cls as in your code, and in a lot of code "in the wild")

Inheritance of metaclass

1 Answers1