Why a recursion happens here?

Question

Recently I read an interesting discussion on how to make a singleton in Python. One of the solutions was a tricky decorator defining a class inside its code as a substitute for decorated class:

def singleton(class_):
    class class_w(class_):
        _instance = None
        def __new__(class2, *args, **kwargs):
            if class_w._instance is None:
                class_w._instance = super(class_w, class2).__new__(class2, *args, **kwargs)
                class_w._instance._sealed = False
            return class_w._instance
        def __init__(self, *args, **kwargs):
            if self._sealed:
                return
            super(class_w, self).__init__(*args, **kwargs)
            self._sealed = True
    class_w.__name__ = class_.__name__
    return class_w

@singleton
class MyClass(object):
    def __init__(self, text):
        print text
    @classmethod
    def name(class_):
        print class_.__name__

x = MyClass(111)
x.name()
y = MyClass(222)
print id(x) == id(y)

Output is:

111     # the __init__ is called only on the 1st time
MyClass # the __name__ is preserved
True    # this is actually the same instance

It is stated, that if we use super(MyClass, self).__init__(text) inside __init__ of MyClass, we get into recursion.

I tested and indeed the recursion happens. But, as I understand, MyClass inherits object, so super(MyClass, self) should just merely be object, but it turns out that super(MyClass, self) is __main__.MyClass

Could you explain what happens here step by step for me to understand the reasons why the recursion happens?

Answer 1

The problem is that by writing super(MyClass, self).__init__(text), you are saying to use the super relative to whatever class MyClass refers to at the time super is called. But the decorator replaces MyClass with a subclass of itself. So when your original __init__ method is called MyClass actually refers to a subclass of the class which defines the executing method.

To say it step by step, I'm going to call the original class (as written in the source) OrigMyClass, and the resulting version (after the decorator) DecMyClass. I'll use MyClass just as a variable, because its meaning changes during the course of execution.

1. You define an __init__ method on OrigMyClass, but that __init__ method calls super(MyClass, self), not super(OrigMyClass, self). Thus, what method will actually be called depends on what MyClass refers to at the time the method is called. The value of MyClass is looked up at execution time like any other variable; placing it inside the super call or inside the __init__ method does not magically bind it to the class it happens to be in when you write it; variables in functions are evaluated when they are called, not when they are defined.

2. The decorator runs. The decorator defines a new class DecMyClass as a subclass of OrigMyClass. DecMyClass defines an __init__ that calls super(DecMyClass, self).

3. After the decorator runs, the name MyClass is bound to the class DecMyClass. Note that this means that when the super(MyClass, self) call later executes, it will be doing super(DecMyClass, self).

4. When you do MyClass(111), you instantiate an object of DecMyClass. DecMyClass.__init__ calls super(DecMyClass, self).__init__. This executes OrigMyClass.__init__.

5. OrigMyClass.__init__ calls super(MyClass, self).__init__. Because MyClass refers to DecMyClass, this is the same as super(DecMyClass, self).__init__. But DecMyClass is a subclass of OrigMyClass. The key point is that because MyClass refers to DecMyClass, OrigMyClass is actually calling super on a subclass of itself.

6. Thus super(DecMyClass, self).__init__ again calls OrigMyClass.__init__, which again calls itself, and so on to infinity.

The effect is the same as this code, which may make the execution path more obvious:

>>> class Super(object):
...     def __init__(self):
...         print "In super init"
...         super(Sub, self).__init__()
>>> class Sub(Super):
...     def __init__(self):
...         print "In sub init"
...         super(Sub, self).__init__()

Note that Super calls super(Sub, self). It is trying to call a superclass method, but it tries to call the superclass method of Sub. The superclass of Sub is Super, so Super winds up calling its own method again.

Edit: Just to clarify the name-lookup issues you raised, here's another slightly different example that has the same result:

>>> class Super(object):
...     def __init__(self):
...         print "In super init"
...         super(someClass, self).__init__()
>>> class Sub(Super):
...     def __init__(self):
...         print "In sub init"
...         super(Sub, self).__init__()
>>> someClass = Sub

This should make it clear that the class argument to super (the first argument, here someClass) is not special in any way. It is just an ordinary name whose value is looked up in the ordinary way at the ordinary time, namely when the super call is executed. As shown by this example, the variable doesn't even have to exist at the time you define the method; the value is looked up at the time you call the method.