I've skimmed several papers by Stefan Brunthaler about something called Quickening that I first found via the Pyston blog, which linked to an ancient bpo issue with a patch created by Stefan (https://bugs.python.org/issue14757).
The gist seems to be to have extra opcodes that only work for certain situations (e.g. INT_BINARY_ADD). In a hot function we can rewrite opcodes with their specialized counterpart. The new opcode contains a guard that rewrites itself back if the guard fails (and then it stays unoptimized).
I think the idea is that it's pretty cheap to keep an extra pointer to a "alternate bytecode" array.