Why there is infinite recursion loop risk in getattribute?

Question

Refer to the second top answer to an existing question: Difference between __getattr__ vs __getattribute__, which including code suggested by someone:

class Count(object):
    def __init__(self, mymin, mymax):
        self.mymin = mymin
        self.mymax = mymax
        self.current = None

    def __getattr__(self, item):
        self.__dict__[item] = 0
        return 0

    def __getattribute__(self, item):
        if item.startswith('cur'):
            raise AttributeError
        return super(Count, self).__getattribute__(item)

obj1 = Count(1, 10)
print(obj1.mymin)
print(obj1.mymax)
print(obj1.current)

My question is:

When I run the code, it did not run into an infinite recursion deep (by ending with maximum recursion depth exceeded). Why? And, if I change the code super(Count, self).__getattribute__(item) to super(object, self).__getattribute__(item), it did run into an infinite loop. Why again?

Please explain the reason with a detailed calling process.

Somehow related: https://stackoverflow.com/questions/2405590/how-do-i-override-getattr-in-python-without-breaking-the-default-behavior — DeepSpace, Oct 09 '18 at 10:45
Unrelated, but updating the object's state - and eventually creating new attributes - on an attribute lookup is a very bad idea (unless your only caching some value in a protected attribute, but that's not the case here) — bruno desthuilliers, Oct 09 '18 at 10:56
"When I run the code, it didn't run into an infinite recursion" => why should it ??? — bruno desthuilliers, Oct 09 '18 at 10:58
@brunodesthuilliers I changed the line "return super(Count, self).__getattribute__(item)" to "return super(object, self).__getattribute__(item) ", and then it did. — CandyCrusher, Oct 09 '18 at 11:01
You should never write `super(object, self)`. Ever. That can never be correct. There's nothing *above* `object`. — Aran-Fey, Oct 09 '18 at 11:02
@Aran-Fey I wonder what will actually happen inside, if I did change? what's the mechanism? — CandyCrusher, Oct 09 '18 at 11:04

Antti Haapala -- Слава Україні · Accepted Answer · 2018-10-09T11:56:31.137

I will try to make it simpler by breaking the self.__dict__[item] into 2 parts:

class Count(object):
    def __getattr__(self, item):
        print('__getattr__:', item)
        d = self.__dict__
        print('resolved __dict__')
        d[item] = 0
        return 0

    def __getattribute__(self, item):
        print('__getattribute__:', item)
        if item.startswith('cur'):
            raise AttributeError
        return super(Count, self).__getattribute__(item)

obj1 = Count()
print(obj1.current)

The output is

__getattribute__: current
__getattr__: current
__getattribute__: __dict__
resolved __dict__
0

Now, if we replace super(Count, self) with the incorrect construct super(object, self) the message is not printed. It is because __getattribute__ will also mask the access to __dict__. However the super object will point to the base class of object which does not exist and hence our __getattribute__ function will always throw AttributeError.

Now, after __getattribute__ fails, __getattr__ is being tried for it ... well, instead of just resolving __dict__ to some value, it tries to get it as an attribute - and ends up calling__getattribute__ again. Hence we get.

....
__getattribute__:  __dict__
__getattr__: __dict__
__getattribute__:  __dict__
__getattr__: __dict__
__getattribute__:  __dict__
__getattr__: __dict__
__getattribute__:  __dict__
__getattr__: __dict__
__getattribute__:  __dict__
__getattr__: __dict__
Traceback (most recent call last):
  File "getattribute.py", line 15, in <module>
    print(obj1.current)
  File "getattribute.py", line 4, in __getattr__
    d = self.__dict__
  File "getattribute.py", line 4, in __getattr__
    d = self.__dict__
  File "getattribute.py", line 4, in __getattr__
    d = self.__dict__
  [Previous line repeated 328 more times]
  File "getattribute.py", line 8, in __getattribute__
    print('__getattribute__: ', item)
RecursionError: maximum recursion depth exceeded while calling a Python object

Had you used setattr(self, item, 0) instead of looking up self.__dict__ this could have been "avoided":

class Count(object):
    def __getattr__(self, item):
        setattr(self, item, 0)
        return 0

    def __getattribute__(self, item):
        if item.startswith('cur'):
            raise AttributeError
        return super(object, self).__getattribute__(item)

obj1 = Count()
print(obj1.current)

of course such code would not have been correct - trying to access any other attribute would have failed nevertheless.

Why __getattribute__ will mask the access to __dict__ ? I tried your code, I thought the code will never reach print statement but it will reach "d = self.__dict__". — CandyCrusher, Oct 09 '18 at 11:41
I saw you added on the explanation, my question completely resolved, thank you! — CandyCrusher, Oct 09 '18 at 11:51
it seems that 'object' doesn't have attribute named __dict__ while any other class inherits it does have. when I use super(Count, self), self.__dict__ can be accessed as __dict__ is an attribute of class Count. Do I understand it correct? — CandyCrusher, Oct 10 '18 at 03:11
Also, what do you mean by "trying to access any other attribute would have failed nevertheless" — CandyCrusher, Oct 10 '18 at 03:15

Why there is infinite recursion loop risk in __getattribute__?

1 Answers1

Why there is infinite recursion loop risk in getattribute?