gh-123748: Add conditional compilation rules to allow HACL SIMD256 to be compiled on universal by freakboy3742 · Pull Request #123989 · python/cpython

Add modifications to the compilation of hashlib to restore the ability to generate a universal build.

#119316 added an implementation of Blake2 to hashlib. That implementation compiles fine on single architecture macOS builds (both x86_64 and ARM64, as verified by CI); but universal2 builds generate a compilation error because the SIMD256 algorithm can be compiled on x86_64, but can't be compiled on ARM64. Since both architectures are compiled in a single pass, autoconf by itself can't detect or control how to include (or exclude) the SIMD256 module.

This PR:

• Adds a Hacl_Hash_Blake2b_Simd256_universal2.c file that conditionally #includes Hacl_Hash_Blake2b_Simd256.c`. This keeps the Python additions isolated, allowing the original HACL sources to be used (and updated) as-is.
• Adds Makefile and configure changes to include the _universal2 wrapper variant if compiling on universal, along with configure logging to identify when this occurs
• Adds a conditional block to the start of blake2module.c that disables the HACL_CAN_COMPILE_SIMD256 symbol if on ARM64. On a pure ARM64 compile, this will never be executed; but in a universal build, it will.

Fixes #123748.

/cc @msprotz

• Issue: Error compiling HACL* Blake2 support for macOS universal binaries #123748

The logic looks good to me, but I wonder if this ends up enabling Blake2s/128 on ARM64? Right now it's not enabled (due to a slow barrel-shifter, I think, which gives so-so performance) by default, but might end up being enabled unintentionally for universal builds.

Also, I don't know if this is specific to this work, but I tried out ./configure --with-builtin-hashlib-hashes=md5,sha1,sha2,sha3,blake2 but then when I tried to make sure built-in hashes were linked properly, I got this:

>>> import hashlib
>>> hashlib.sha256
<built-in function openssl_sha256>

(not sure this is related to the problem of universal builds, or if this is a more general issue with configure)

The logic looks good to me, but I wonder if this ends up enabling Blake2s/128 on ARM64? Right now it's not enabled (due to a slow barrel-shifter, I think, which gives so-so performance) by default, but might end up being enabled unintentionally for universal builds.

It will - unlike SIMD256.c, SIMD128.c can be compiled on ARM64. However, given the existing configuration, it wouldn't be included in an ARM64-only build. I'll push an update to do a similar "#include and #undef" trick for the SIMD128 implementation so that we retain parity with single-architecture builds.

Also, I don't know if this is specific to this work, but I tried out ./configure --with-builtin-hashlib-hashes=md5,sha1,sha2,sha3,blake2 but then when I tried to make sure built-in hashes were linked properly, I got this:

>>> import hashlib
>>> hashlib.sha256
<built-in function openssl_sha256>

(not sure this is related to the problem of universal builds, or if this is a more general issue with configure)

This is the same behavior I see in the official universal build of Python 3.13.0rc2 running on macOS. The same is true of other older Pythons that I've tested back to 3.8, so it doesn't appear to be specific to this patch.

@msprotz Are you happy with the current state of this patch?