bpo-41100: Support macOS 11 and Apple Silicon by ronaldoussoren · Pull Request #22855 · python/cpython

We should consider just dropping support for macOS 10.4 here
to simplify the code.

This is support for ctypes on macOS/arm64 based
on PR 21249 by Lawrence D'Anna (Apple).

Changes:
- changed __builtin_available tests from 11.0 to 10.15
- added test to setup.py for ffi_closure_alloc and use
  that in malloc_closure.c
- Minor change in the code path for ffi_prep_closure_var
  (coding style change)

The preprocessor guard in the old version doesn't work, and
isn't really needed (10.4 only supported 32-bit code where
unsigned long is the same as uint32_t).

This also adds an option to stop building after compiling the
3th-party dependencies, as well as a script for archiving those
dependencies. This makes it easier to work on the build.

There are three changes to build-installer.py related to universal2
support:

1. Add 'universal2' information to build-installer.py;
2. Building OpenSSL for arm64 requires a patch at this time;
3. For some reason I had to patch the Tcl build to avoid a build error.

Needed because my previous workaround doesn't work anymore.

This uses a private API, that should be made public later...

- __builtin_available is not present in the compiler, fall
  back to the ancient way of testing if a symbol is usable.
- Logic error code in pytime.c resulted in a compile error.

…oren/cpython into macos11-deploy-earlier-branch

The tests are not entirely done yet, and something
similar needs to be done for the time module.

On Big Sur, there is no longer a separate dylib file for each system
library. Instead, they have been combined into a single shared cache
file. As a result, it no longer suffices to check file existence to
determine whether a system library is present. Fix this by calling
_dyld_shared_cache_contains_path to check the dyld shared cache as well.

The changes under lib-python/ are copied from
python/cpython#22855.

The _dyld_shared_cache_contains_path function is exposed to the ctypes
python code using CFFI, similar to what is done in other modules, such
as lib_pypy/_sha3.

Fixes issue #3314.

Co-authored-by:  Lawrence D’Anna <lawrence_danna@apple.com>

* Add support for macOS 11 and Apple Silicon (aka arm64)
   
  As a side effect of this work use the system copy of libffi on macOS, and remove the vendored copy

* Support building on recent versions of macOS while deploying to older versions

  This allows building installers on macOS 11 while still supporting macOS 10.9.

pythonGH-25806 was a python 3.8 backport of pythonGH-22855 to support macOS 11,
but didn't backport the changes that enabled building python
with a new macOS SDK and target an older macOS version.
This commit adds support for that.

…295)

* [3.9] bpo-41100: Support macOS 11 and Apple Silicon (GH-22855)

Co-authored-by:  Lawrence D’Anna <lawrence_danna@apple.com>

* Add support for macOS 11 and Apple Silicon (aka arm64)

  As a side effect of this work use the system copy of libffi on macOS, and remove the vendored copy

* Support building on recent versions of macOS while deploying to older versions

  This allows building installers on macOS 11 while still supporting macOS 10.9..
(cherry picked from commit 4176193)

Co-authored-by: Ronald Oussoren <ronaldoussoren@mac.com>

* Back port of changes to _decimal to support arm64

* temp_dir is in test.support in 3.9

It was an ancient, modified copy of libffi that has not been in use
since pythonGH-22855.

It has had no effect on non-macOS platforms for a long time, and has had
the non-obvious effect of invoking `pkg_config` and not setting
`-DUSING_APPLE_OS_LIBFFI` on macOS since pythonGH-22855.

…00544)

It has had no effect on non-macOS platforms for a long time, and has had
the non-obvious effect of invoking `pkg_config` and not setting
`-DUSING_APPLE_OS_LIBFFI` on macOS since GH-22855.

It was an ancient, modified copy of libffi that has not been in use
since GH-22855.

(Apple Silicon being their aarch64 platform.)

Patches consist of:
 - Upstream work: python/cpython#22855
 - Fix for setup.py to find libbz2.tbd and libz.tbd now that with the
   shared library cache there's nothing in /usr/lib.
   See: https://bugs.python.org/issue41116
 - Addition of __arch64__ case to fix _decimal module. A very similar fix
   has since been committed upstream.

(Apple Silicon being their aarch64 platform.)

This is backport of the same in lang/python39. Only minor changes were
needed.

Patches consist of:
 - Upstream work: python/cpython#22855
 - Fix for setup.py to find libbz2.tbd and libz.tbd now that with the
   shared library cache there's nothing in /usr/lib.
   See: https://bugs.python.org/issue41116
 - Addition of __arch64__ case to fix _decimal module. A very similar fix
   has since been committed upstream.

(Apple Silicon being their aarch64 platform.)

This is backport of the same in lang/python39 and lang/python38. Some
parts weren't applicable in 3.7. The setup.py script needed some work
on the ffi code. Otherwise, minor changes.

Patches consist of:
 - Upstream work: python/cpython#22855
 - Fix for setup.py to find libbz2.tbd and libz.tbd now that with the
   shared library cache there's nothing in /usr/lib.
   See: https://bugs.python.org/issue41116
 - Addition of __arch64__ case to fix _decimal module. A very similar fix
   has since been committed upstream.

(Apple Silicon being their aarch64 platform.)

Patches consist of:
 - Upstream work: python/cpython#22855
 - Fix for setup.py to find libbz2.tbd and libz.tbd now that with the
   shared library cache there's nothing in /usr/lib.
   See: https://bugs.python.org/issue41116
 - Addition of __arch64__ case to fix _decimal module. A very similar fix
   has since been committed upstream.

(Apple Silicon being their aarch64 platform.)

This is backport of the same in lang/python39. Only minor changes were
needed.

Patches consist of:
 - Upstream work: python/cpython#22855
 - Fix for setup.py to find libbz2.tbd and libz.tbd now that with the
   shared library cache there's nothing in /usr/lib.
   See: https://bugs.python.org/issue41116
 - Addition of __arch64__ case to fix _decimal module. A very similar fix
   has since been committed upstream.

(Apple Silicon being their aarch64 platform.)

This is backport of the same in lang/python39 and lang/python38. Some
parts weren't applicable in 3.7. The setup.py script needed some work
on the ffi code. Otherwise, minor changes.

Patches consist of:
 - Upstream work: python/cpython#22855
 - Fix for setup.py to find libbz2.tbd and libz.tbd now that with the
   shared library cache there's nothing in /usr/lib.
   See: https://bugs.python.org/issue41116
 - Addition of __arch64__ case to fix _decimal module. A very similar fix
   has since been committed upstream.