One important piece is that regrtest currently has no tests (e.g. there is no test_regrtest.py), so changing it must be done more carefully. How do people feel about new (or newly modified) regrtest classes and functions going into a different fully-tested module with its own test_* file. Currently, regrtest is a mixture of definitions and "running code," so I would be leery of adding test_regrtest.
This could also be done by making regrtest a package, so that regrtest-related modules would be in a single directory.