Can you explain how does soft shutdown work in more details?

The base concept is, after Py_Initialize, save all GC objects and modules, and reset them when you shutting down.
Since the domain will be unload, the saved state cannot use any C# data types, also we need a dummy GC for storing the old objects for make sure some dealloc operations can call _PyObject_GC_UnTrack macro without a nullptr error. You can see the details in runtime_state.cs.
After discuss with @mjmvisser, he points some problems that relate restart Python will crash the PyQt. So I try to make a solution for that. I was intend to create two modes, one for reset all GC objects and modules(for now) and one for reset all slots of CLR types after previous mode, but I just saw this issue(#957), so I push it in advance.
Well, although my computer pass all tests, but the CI still not, I still need some fixes for the CI.

I'm unsure how this relates to #957. Seems to be more about improving the GC handling?

The prototype that @BadSingleton did was, in implementation, fairly simple (took us a while to figure it out of course):

1. Don't Py_Finalize in Shutdown
2. On Shutdown, go through the TypeManager and set all the slots to zero (except the gc ones).

Well, Python crash is connect with GC handling, after domain unload, when the GC collect occurred, it will traverse all GC objects, but the memory from old domain has been invalid, any access will result to crash.
If you just take them from GC chains, you can avoid access those invalid memory access to prevent crash. It maybe simple, but lots of memory will leak. So I need to do a lot of work to reduce the leaking, but it still cannot avoid them at all.

I've summarized my understanding in this document: https://docs.google.com/document/d/1a9OLsdKHXJ6MxHjo0WlVRcfjAiaP5hONNKUKKH6Tr-o

I see two PRs that would be easy to pull out of amos' code and check in right now:

• The SlotsHolder, which implements the proposal in Python can't reliably re-initialize; new proposal for Unity's domain reload #957
• The __import__ restoration.
  They don't (shouldn't) break anything so we can merge them in.

I don't yet understand what the gc chains thing is trying to address. I also don't understand how amos handles the modules. I suspect they're related.

It seems like amos is turning off python gc for the proxy objects. If that works, we should just turn it all off and then we can eliminate the native code page. But that's yet another PR.

There seems to be some refcount fixes as well. If we can get those fixes in separate PRs that makes it easier to review.

@BadSingleton / @amos402 can you please merge from master?

Codecov Report

Merging #958 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #958   +/-   ##
=======================================
  Coverage   86.25%   86.25%           
=======================================
  Files           1        1           
  Lines         291      291           
=======================================
  Hits          251      251           
  Misses         40       40           

Flags with carried forward coverage won't be shown. Click here to find out more.

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c79be84...8d00e4c. Read the comment docs.