gh-107862: Add property-based round-trip tests for base64 by encukou · Pull Request #119406 · python/cpython

I thought there had been a decision not to add hypothesis tests to any module not under active development. During development, hypothesis is good at teasing out special cases that need to be fixed and have an explicit targeted test. Once in production, hypothesis doesn't do much for us and leaves it unclear whether the search space for a property has actually been covered.

AFAICT this PR doesn't add any value, it just eats clock cycles and adds another tool that a person has to learn before they can read the tests. If our current tests are inadequate in any way, we should add specifically targeted tests to fill the gaps.

We should differentiate a goal we care about, "thoroughly understand and test our code" from the more dubious goal, "Let's use this tool every we can." ISTM that this PR only does the latter and doesn't help base64 in any way.

Also note, that we've already had a number of people do the exercise of trying hypothesis on many standard library modules. We were able to do that without actually checking in the tests into main distribution. FWIW, the main result of the exercise was identifying bugs in the tester's understanding of the module (i.e. an expectation that colorsys conversions would round-trip without understanding gamut limits and published specifications that did not round-trip).