◐ Shell
clean mode source ↗

gh-103583: Isolate CJK codec modules by erlend-aasland · Pull Request #103869 · python/cpython

Lifetime should be guaranteed, since each MultibyteCodecObjects owns a
reference the cjk codec module that owns the memory of the
MultibyteCodec struct.

@erlend-aasland

This approach exploits the dependency introduced in gh-103589; we know for sure that the _codec* extension modules will outlive _multibytecodec. This means we can store the cjk codec module state safely in the codec struct and modify the various codec handlers to accept the codec struct (iso. void *config). Then we add custom cjk module state where needed, and conveniently fetch it from the codec struct. (Quick and dirty explanation; see the code if what I wrote does not make sense 😎)

@bedevere-bot

🤖 New build scheduled with the buildbot fleet by @erlend-aasland for commit ef983c4 🤖

If you want to schedule another build, you need to add the 🔨 test-with-refleak-buildbots label again.

@erlend-aasland

BTW, I removed the initialized guards. I'm pretty sure we can do that, but I haven't verified it yet.

@erlend-aasland

Also, we can split out cb2c9e7 as a separate PR. That may be beneficial regarding to future bisects; it will also make this PR easier to review.

Similarly, f6ed5df can be split up to do the arg spec changes first, and then move stuff to module state afterwards.

@erlend-aasland

@corona10

@erlend-aasland

(Pulling in main to get the latest ref. leak fixes by Jelle)

@bedevere-bot

🤖 New build scheduled with the buildbot fleet by @erlend-aasland for commit dec79a7 🤖

If you want to schedule another build, you need to add the 🔨 test-with-refleak-buildbots label again.

@erlend-aasland erlend-aasland changed the title gh-103583: Isolate CJK modules, alternative 2 gh-103583: Isolate CJK codec modules

Apr 26, 2023

This was referenced

Apr 26, 2023

corona10

corona10

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, This is quite a complicated but nice approach. But nothing can be better than this.
Thanks for the hard work!

@erlend-aasland

Thanks for the review, Dong-hee! Yes, it is complicated, but as shown in my competing PRs, other approaches are even more complicated.

@vstinner

@erlend-aasland