gh-90548: Make musl test skips smarter (fixes Alpine errors)#131313

bitdancer

Make musl test skips smarter (fixes Alpine errors)

A relatively small number of tests fail when the underlying c library is
provided by musl. This was originally reported in bpo-46390 by
Christian Heimes. Among other changes, these tests were marked for
skipping in gh-31947/ef1327e3 as part of bpo-40280 (emscripten support),
but the skips were conditioned on the platform being emscripten (or
wasi, skips for which ere added in 9b50585).

In gh-131071 Victor Stinner added a linked_to_musl function to enable
skipping a test in test_math that fails under musl, like it does on a
number of other platforms. This check can successfully detect that
python is running under musl on Alpine, which was the original problem
report in bpo-46390.

This PR replaces Victor's solution with an enhancement to
platform.libc_ver that does the check more cheaply, and also gets the
version number. The latter is important because the math test being
skipped is due to a bug in musl that has been fixed, but as of this
checkin date has not yet been released. When it is, the test skip can
be fixed to check for the minimum needed version.

The enhanced version of linked_to_musl is also used to do the skips of
the other tests that generically fail under musl, as opposed to
emscripten or wasi only failures. This will allow these tests to be
skipped automatically on Alpine.

This PR does not enhance libc_ver to support emscripten and wasi, as
I'm not familiar with those platforms; instead it returns a version
triple of (0, 0, 0) for those platforms. This means the musl tests will
be skipped regardless of musl version, so ideally someone will add
support to libc_ver for these platforms.

📚 Documentation preview 📚: https://cpython-previews--131313.org.readthedocs.build/

Issue: Multiple test failures on Alpine 3.15 / musl-1.2.2-r7 #90548

A relatively small number of tests fail when the underlying c library is provided by musl. This was originally reported in bpo-46390 by Christian Heimes. Among other changes, these tests were marked for skipping in pythongh-31947/ef1327e3 as part of bpo-40280 (emscripten support), but the skips were conditioned on the *platform* being emscripten (or wasi, skips for which ere added in 9b50585). In pythongh-131071 Victor Stinner added a linked_to_musl function to enable skipping a test in test_math that fails under musl, like it does on a number of other platforms. This check can successfully detect that python is running under musl on Alpine, which was the original problem report in bpo-46390. This PR replaces Victor's solution with an enhancement to platform.libc_ver that does the check more cheaply, and also gets the version number. The latter is important because the math test being skipped is due to a bug in musl that has been fixed, but as of this checkin date has not yet been released. When it is, the test skip can be fixed to check for the minimum needed version. The enhanced version of linked_to_musl is also used to do the skips of the other tests that generically fail under musl, as opposed to emscripten or wasi only failures. This will allow these tests to be skipped automatically on Alpine. This PR does *not* enhance libc_ver to support emscripten and wasi, as I'm not familiar with those platforms; instead it returns a version triple of (0, 0, 0) for those platforms. This means the musl tests will be skipped regardless of musl version, so ideally someone will add support to libc_ver for these platforms.

In adding tests for the new platform code I found a bug in the old code: if a valid version is passed for version and it is greater than the version found for an so *and* there is no glibc version, then the version from the argument was returned. The code changes here fix that, as well as fixing my own broken additions.

vstinner

The code detecting the musl version works for me on Alpine Linux:

~/cpython # LD_LIBRARY_PATH=$PWD ./python 
Python 3.14.0a5+ (heads/main-dirty:fe186d76cfd, Mar 11 2025, 08:54:51) [GCC 14.2.0] on linux
>>> import test.support
>>> test.support.linked_to_musl()
(1, 2, 5)


~/cpython # LD_LIBRARY_PATH=$PWD ./python -m test.pythoninfo|grep libc
platform.libc_ver: musl 1.2.5


~/cpython # apk info musl
musl-1.2.5-r9 description:
the musl c library (libc) implementation

musl-1.2.5-r9 webpage:
https://musl.libc.org/

musl-1.2.5-r9 installed size:
646 KiB

picnixz

For my suggestions with dedents, please check them manually as I'm not entirely sure that I didn't mix tabs and spaces.

picnixz

Btw, do we want to create an issue (maybe an umbrella one for musl issues) or do we make it a skip-issue?

bitdancer

There is an issue (gh-90548). I forgot the gh issue existed when I mentioned the bpo issue :( I'm not actually sure how one does the link, as I'm still coming up to speed on the "modern" way of doing development ;)

picnixz

I'm not actually sure how one does the link, as I'm still coming up to speed on the "modern" way of doing development ;)

No worries! the PR title should be gh-XXXXXX: title or GH-XXXXXX: title and the bot would link it.

vstinner

LGTM, but please fix the important typo 'musl' in binary.

picnixz

As I said, I won't block the PR for cosmetics only though I would have aligned with the surrounding/existing code when possible.

serhiy-storchaka

LGTM in general.

This didn't cause a test failure because the broken logic caused the regex to always run...which is a lot slower.

bitdancer

I'm thinking this is not worth backporting?

vstinner

I'm thinking this is not worth backporting?

I don't think that it's worth it to backport this change to stable branches. Supporting Alpine Linux can be seen as a new feature ;-)

bitdancer added 3 commits March 15, 2025 15:17

Add docs, including some existing is_xxx's.

417143e

bitdancer requested a review from pganssle as a code owner March 15, 2025 22:31

bitdancer self-assigned this Mar 15, 2025

bitdancer requested a review from abalkin as a code owner March 15, 2025 22:31

bedevere-app Bot added the awaiting core review label Mar 15, 2025

bitdancer changed the title ~~better musl test checks~~ Mar 15, 2025

bitdancer requested review from tiran and vstinner March 15, 2025 22:40

bitdancer mentioned this pull request Mar 17, 2025

Multiple test failures on Alpine 3.15 / musl-1.2.2-r7 #90548

Closed

vstinner mentioned this pull request Mar 17, 2025

test_re fails on Alpine Linux (with musl) #131342

Closed

vstinner reviewed Mar 17, 2025

View reviewed changes

vstinner added the skip news label Mar 17, 2025

picnixz reviewed Mar 17, 2025

View reviewed changes

picnixz changed the title ~~Make musl test skips smarter (fixes Alpine errors)~~ Mar 17, 2025

bitdancer added 3 commits March 17, 2025 14:24

Address vstinner review comments.

ef36eb2

Test some version number variants we accept.

909ea3b

Add news item about libc_ver enhancement.

4e76d78

vstinner approved these changes Mar 18, 2025

View reviewed changes

bedevere-app Bot added awaiting merge and removed awaiting core review labels Mar 18, 2025

picnixz approved these changes Mar 18, 2025

View reviewed changes

serhiy-storchaka approved these changes Mar 18, 2025

View reviewed changes

Fix 'in' phrase logic. …

83add5d

This didn't cause a test failure because the broken logic caused the regex to always run...which is a lot slower.

Prettify re expression using re.VERBOSE.

4d6aa92

Use more verbose style rather than :=

111e2ec

serhiy-storchaka approved these changes Mar 19, 2025

View reviewed changes

bitdancer merged commit 6146295 into python:main Mar 19, 2025

bedevere-app Bot removed the awaiting merge label Mar 19, 2025

This was referenced Apr 28, 2025

gh-87414: add musl support to platform.libc_ver #103784

Closed

platform.libc_ver() returns no information on Alpine Linux #87414

Closed

bitdancer deleted the better_musl_test_checks branch May 26, 2025 17:06

cdce8p mentioned this pull request Aug 16, 2025

gh-90548: Fix musl version detection with --strip-all #137864

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gh-90548: Make musl test skips smarter (fixes Alpine errors)#131313

gh-90548: Make musl test skips smarter (fixes Alpine errors)#131313
bitdancer merged 9 commits into
python:mainfrom
bitdancer:better_musl_test_checks

bitdancer commented Mar 15, 2025 •

edited by bedevere-app Bot

Loading

Uh oh!

vstinner commented Mar 17, 2025

Uh oh!

picnixz left a comment

Uh oh!

picnixz commented Mar 17, 2025

Uh oh!

bitdancer commented Mar 17, 2025

Uh oh!

picnixz commented Mar 17, 2025

Uh oh!

vstinner left a comment

Uh oh!

picnixz left a comment

Uh oh!

serhiy-storchaka left a comment

Uh oh!

bitdancer commented Mar 19, 2025

Uh oh!

vstinner commented Mar 19, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

bitdancer commented Mar 15, 2025 • edited by bedevere-app Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vstinner commented Mar 17, 2025

Uh oh!

picnixz left a comment

Choose a reason for hiding this comment

Uh oh!

picnixz commented Mar 17, 2025

Uh oh!

bitdancer commented Mar 17, 2025

Uh oh!

picnixz commented Mar 17, 2025

Uh oh!

vstinner left a comment

Choose a reason for hiding this comment

Uh oh!

picnixz left a comment

Choose a reason for hiding this comment

Uh oh!

serhiy-storchaka left a comment

Choose a reason for hiding this comment

Uh oh!

bitdancer commented Mar 19, 2025

Uh oh!

vstinner commented Mar 19, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

bitdancer commented Mar 15, 2025 •

edited by bedevere-app Bot

Loading