This looks like it's ready, or very close. @maxbachmann, @eryksun is there anything left to do here? I don't mind it being done in future PRs if they're necessary.

I am done with everything I planned for this PR. As suggested by @eryksun I plan to extend the sys module to retrieve the api partition. I will add this in a separate PR.

I plan to extend the sys module to retrieve the api partition. I will add this in a separate PR.

Give that a new issue as well. We really ought to add the platform to sys.implementation as well, since currently it's a bit weird to check for 32-bit vs 64-bit vs ARM64 as well (sys.winver is the only way). I assume that would be where the API partition is indicated, too.

Okay, thanks for holding. What I'm hearing from my colleagues at Microsoft is that the PathCch* APIs are implemented in the Xbox OS, just not exposed through the partition or their import lib. It sounds like both of these will be fixed for an update later this year, at which point our original code will be fine. The only restriction seems to be that the APIs are not implemented on Win7, so even though games are meant to be compatible all the way back, if we were to use the API then it would not work.

Until the GDK update arrives, it seems like we can probably use code like what we used to have to load the API dynamically (from getpathp.c in the 3.7 branch):

static int _PathCchCombineEx_Initialized = 0;
typedef HRESULT(__stdcall *PPathCchCombineEx) (PWSTR pszPathOut, size_t cchPathOut,
                                               PCWSTR pszPathIn, PCWSTR pszMore,
                                               unsigned long dwFlags);
static PPathCchCombineEx _PathCchCombineEx;


static void
join(wchar_t *buffer, const wchar_t *stuff)
{
    if (_PathCchCombineEx_Initialized == 0) {
        HMODULE pathapi = LoadLibraryExW(L"api-ms-win-core-path-l1-1-0.dll", NULL,
                                         LOAD_LIBRARY_SEARCH_SYSTEM32);
        if (pathapi) {
            _PathCchCombineEx = (PPathCchCombineEx)GetProcAddress(pathapi, "PathCchCombineEx");
        }
        else {
            _PathCchCombineEx = NULL;
        }
        _PathCchCombineEx_Initialized = 1;
    }


    if (_PathCchCombineEx) {
        if (FAILED(_PathCchCombineEx(buffer, MAXPATHLEN+1, buffer, stuff, 0))) {
            Py_FatalError("buffer overflow in getpathp.c's join()");
        }
    } else {
        if (!PathCombineW(buffer, buffer, stuff)) {
            Py_FatalError("buffer overflow in getpathp.c's join()");
        }
    }
}

@maxbachmann Would you be able to give this approach a try and see if it works? Unfortunately, I don't think there's any way to detect the version of the games SDK involved, so we'd just have to keep this until we assume everyone's on the fixed update.

If it helps (and I suspect it won't), it also ought to be okay to temporarily define the PARTITION constants needed when including pathcch.h. It's more likely going to be better to exclude the include statement in GAMES when we're defining a GetProcAddress wrapper, since that way we can define our own PathCchCombineEx implementation and it shouldn't collide with the real one until we choose to let it. You can probably even keep the existing implementation as a fallback for the (hopefully rare) cases where games may be running on Win7, but I'm happy to defer to you on that one, as you've likely got a better view of the gamedev landscape than I do.

Okay, thanks for holding. What I'm hearing from my colleagues at Microsoft is that the PathCch* APIs are implemented in the Xbox OS, just not exposed through the partition or their import lib. It sounds like both of these will be fixed for an update later this year, at which point our original code will be fine.

Yes I did read in the developer forums, that there is already an implementation as well. Great to hear that they are likely going to get exposed at some point.

Would you be able to give this approach a try and see if it works? Unfortunately, I don't think there's any way to detect the version of the games SDK involved, so we'd just have to keep this until we assume everyone's on the fixed update.

just gave this a quick test and it works both on the Xbox One and Xbox Scarlett.

You can probably even keep the existing implementation as a fallback for the (hopefully rare) cases where games may be running on Win7, but I'm happy to defer to you on that one, as you've likely got a better view of the gamedev landscape than I do.

I do not think we need to take Windows 7 into consideration here:

1. Cpython overall does not support Windows 7 anymore
2. when building games for the PC you can always build against the desktop partition. Right now you only really need to use the games partition on the xbox, which does not use Windows 7 anyways.
3. even on PC the share of Windows 7 users is really slim. According to the current steam hardware survey around 1.5% of Windows users is on Windows 7: https://store.steampowered.com/hwsurvey/directx/

!buildbot .Windows.

🤖 New build scheduled with the buildbot fleet by @zooba for commit 331b1f4 🤖

The command will test the builders whose names match following regular expression: .*Windows.*

The builders matched are:

• AMD64 Windows10 PR
• ARM64 Windows Non-Debug PR
• AMD64 Windows10 Pro PR
• ARM64 Windows PR

There we go. I expect there'll be no buildbot issues, but worth double checking (there are some unrelated stack issues, so may still fail, but we can ignore those).

Other than that, I'm happy to merge this whenever. @eryksun I'll wait for a positive signal from you.

Thanks @maxbachmann! This was a huge effort, but I think it's worth it. Don't shy away from adjusting things we decided here if it seems important - nothing we changed should count as a public API change, so we can revert or modify if needed.