Like other ABCs, contextlib.AbstractContextManager should support the pattern where setting a method to None disables structural subtyping, which was introduced across the standard library in issue 25958. Ivan Levkivskyi suggested making AbstractContextManager support this pattern in CR comments in https://github.com/python/cpython/pull/1412/files#r114482765.
Should this change be applied to 3.6 as well as master? I'm leaning towards yes, because the resolution to issue 25958 (https://hg.python.org/cpython/rev/72b9f195569c) added a general statement in the documentation that "Setting a special method to ``None`` indicates that the corresponding operation is not available". Thus, the fact that contextlib.AbstractContextManager doesn't obey this rule is a bug.
I'll send a PR shortly.