◐ Shell
clean mode source ↗

gh-92810: Reduce memory usage by ABCMeta.__subclasscheck__ (alternative2) by dolfinus · Pull Request #150540 · python/cpython

Alternative implementation of #131914 which does not check __subclasses__ recursively.
To handle cases like:

class Number(ABC): ...

class Real(Number): ...

class Integral(Real): ...

Integral.register(int)

assert issubclass(int, Number) is True

which previously were implemented via recursive __subclasses__ check, method cls.register(subclass) is calling super(cls).register(subclass) recursively ("bubble-up" registration to all the parents).

For benchmark from #131914:

sudo ./python -m pyperf system tune
taskset -c 0 ./python benchmark.py --metaclass abc.ABCMeta --rounds 3 --classes 5000
taskset -c 0 ./python benchmark.py --metaclass _py_abc.ABCMeta --rounds 3 --classes 5000
Impl Max memory before, MB Max memory after, MB
_abc 6331 50
_py_abc 4422 61
Impl Total time before Total time after
_abc 6m 16s 1m 23s
_py_abc 8m 48s 3m 53s
Check Impl before after Impl before after
isinstance(child, Parent) _abc 0.115us
4MiB...15MiB
0.121us
2MiB...15MiB
_py_abc 0.212us
11MiB...24MiB
0.218us
11MiB...24MiB
issubclass(Child, Parent) _abc 0.105us
0MiB...1MiB
0.111us
0MiB...1MiB
_py_abc 0.203us
6MiB...8MiB
0.208us
5MiB...8MiB
isinstance(child, Grandparent) _abc 0.112us
0MiB...2MiB
0.119us
1MiB...1MiB
_py_abc 0.210us
4MiB...7MiB
0.216us
4MiB...7MiB
issubclass(Child, Grandparent) _abc 0.103us
0MiB
0.108us
0MiB
_py_abc 0.201us
0MiB...1MiB
0.207us
0MiB...1MiB
not isinstance(child, Sibling) _abc 0.113us
4MiB...14MiB
0.121us
3MiB...14MiB
_py_abc 0.348us
13MiB...23MiB
0.354us
13MiB...22MiB
not issubclass(Child, Sibling) _abc 0.105us
1MiB
0.110us
1MiB...2MiB
_py_abc 0.328us
8MiB...10MiB
0.332us
9MiB...11MiB
not isinstance(child, Cousin) _abc 0.115us
1MiB...2MiB
0.121us
1MiB...2MiB
_py_abc 0.350us
7MiB...9MiB
0.354us
7MiB...10MiB
not issubclass(Child, Cousin) _abc 0.104us
0MiB
0.110us
0MiB...1MiB
_py_abc 0.329us
4MiB
0.333us
4MiB...5MiB
not isinstance(child, Uncle) _abc 7.268us
6174MiB...6333MiB
0.125us
0MiB...1MiB
_py_abc 9.957us
4382MiB...4422MiB
0.360us
6MiB
not issubclass(Child, Uncle) _abc 7.099us
6171MiB
0.114us
0MiB
_py_abc 9.936us
4380MiB
0.336us
4MiB...5MiB

Memory increment is measured during isinstance() / issubclass() calls, not during preparation, like class creation or registration where actual registry allocation is performed. So memory usage in tables below is almost always 0.

Timing drop in _py_abc implementation for 2 first rows is due to if subclass in cls._abc_registry: check added to match _abc.c implementation.

Check Impl before after Impl before after
isinstance(child, Parent.register) _abc 0.273us
0MiB
0.272us
0MiB
_py_abc 0.440us
0MiB
0.275us
0MiB
issubclass(Child, Parent.register) _abc 0.154us
0MiB
0.157us
0MiB
_py_abc 0.427us
0MiB
0.261us
0MiB
isinstance(child, Grandparent.register) _abc 0.114us
0MiB
0.120us
0MiB
_py_abc 0.253us
0MiB
0.262us
0MiB
issubclass(Child, Grandparent.register) _abc 0.103us
0MiB
0.109us
0MiB
_py_abc 0.240us
0MiB
0.247us
0MiB
not isinstance(child, Sibling.register) _abc 0.027us
0MiB
0.030us
1MiB
_py_abc 0.028us
0MiB
0.029us
2MiB
not issubclass(Child, Sibling.register) _abc 0.018us
0MiB
0.018us
1MiB
_py_abc 0.018us
0MiB
0.018us
2MiB
not isinstance(child, Cousin.register) _abc 0.028us
0MiB
0.028us
2MiB
_py_abc 0.028us
0MiB
0.029us
3MiB
not issubclass(Child, Cousin.register) _abc 0.018us
0MiB
0.018us
2MiB
_py_abc 0.019us
0MiB
0.018us
3MiB
not isinstance(child, Uncle.register) _abc 0.249us
0MiB
0.233us
2MiB
_py_abc 0.843us
0MiB
0.866us
3MiB
not issubclass(Child, Uncle.register) _abc 0.238us
0MiB
0.227us
2MiB
_py_abc 0.815us
0MiB
0.839us
3MiB

Just to check that nothing is broken:

Check Impl before after Impl before after
not isinstance(child, Unrelated) _abc 0.028us
0MiB
0.028us
0MiB
_py_abc 0.028us
0MiB
0.029us
0MiB
not issubclass(Child, Unrelated) _abc 0.018us
0MiB
0.018us
0MiB
_py_abc 0.018us
0MiB
0.018us
0MiB
not isinstance(child, UnrelatedABC) _abc 0.118us
0MiB
0.121us
0MiB
_py_abc 0.469us
0MiB
0.466us
0MiB
not issubclass(Child, UnrelatedABC) _abc 0.110us
0MiB
0.112us
0MiB
_py_abc 0.442us
0MiB
0.442us
0MiB

Flamegraphs for _py_abc impl and test issubclass_uncle (the most time and memory consuming case on main):
main_vs_pr150540.tar.gz