`_abc._abc_subclasscheck` has very poor performance and (I think) a memory leak

samuelcolvin

Bug report

I'm been hunting a memory leak in pydantic and I keep seeing _abc_subclasscheck popping up in the memray flamegraph.

Locally I've seen memray report that _abc_subclasscheck is using 2.8GB of memory in some cases!

I can't get anything that bad in a minimal example, but I have the following:

from abc import ABCMeta
from datetime import datetime


class MyMetaclass(ABCMeta):
    pass


class MyClass(metaclass=MyMetaclass):
    pass


def main():
    class Foobar(MyClass):
        pass

    assert issubclass(Foobar, MyClass)
    assert not issubclass(int, MyClass)
    assert not issubclass(str, MyClass)
    assert not issubclass(datetime, MyClass)
    t = type('A', (), {})
    assert not issubclass(t, MyClass)


if __name__ == '__main__':
    import os, psutil
    process = psutil.Process(os.getpid())
    mb = 1024 * 1024
    last = 0
    for i in range(5_000):
        main()
        # mem = process.memory_info().rss
        # print(f'{i + 1:>4d} {mem / mb:8.2f}MB {(mem - last) / mb:+8.2f}MB | {"━" * int(mem / 8_000_000)}')
        # last = mem

A few things to note:

the commented out last few lines print current memory and change in memory over time
I'm not sure exactly which issubclass calls are necessary, certainly I don't see a rise in memory with just the last one
some memory drops (presumably related to clearing the abc caches?) is happening, but overall memory is increasing
The performance of issubclass on an abc is very poor indeed - this script takes 14seconds, if i switch ABCMeta to type it takes 121ms!

Your environment

Python 3.10.0 installed with venv
Ubuntu 21.10

`_abc._abc_subclasscheck` has very poor performance and (I think) a memory leak

Linked PRs