◐ Shell
reader mode source ↗
Skip to content

gh-115119: Default to --with-system-libmpdec=yes#118539

Merged
erlend-aasland merged 14 commits into
python:mainfrom
erlend-aasland:libmpdec-default-to-system-lib
May 6, 2024
Merged

gh-115119: Default to --with-system-libmpdec=yes#118539
erlend-aasland merged 14 commits into
python:mainfrom
erlend-aasland:libmpdec-default-to-system-lib

Conversation

@erlend-aasland

@erlend-aasland erlend-aasland commented May 3, 2024 •
edited
Loading

Copy link
Copy Markdown
Contributor

Mark _decimal as missing if libmpdecimal is not found.

📚 Documentation preview 📚: https://cpython-previews--118539.org.readthedocs.build/

@erlend-aasland
pythongh-115119: Default to --with-system-libmpdec=yes …
268d270 
Mark _decimal as missing if libmpdecimal is not found.
@bedevere-app bedevere-app Bot mentioned this pull request May 3, 2024
Closed
15 tasks
@erlend-aasland

erlend-aasland commented May 3, 2024 •
edited
Loading

Copy link
Copy Markdown
Contributor Author 
$ ./configure --with-system-libmpdec=no | grep -E "(decimal|mpdec)"
checking for --with-system-libmpdec... no
checking for --with-decimal-contextvar... yes
checking for decimal libmpdec machine... universal
checking for stdlib extension module _decimal... yes
$ ./configure | grep -E "(decimal|mpdec)"
checking for --with-system-libmpdec... yes
checking for libmpdec... yes
checking for --with-decimal-contextvar... yes
checking for stdlib extension module _decimal... yes
$ grep -E "_DECIMAL_(CFLAGS|LDFLAGS)=" Makefile
MODULE__DECIMAL_CFLAGS=-I/opt/homebrew/Cellar/mpdecimal/4.0.0/include
MODULE__DECIMAL_LDFLAGS=-L/opt/homebrew/Cellar/mpdecimal/4.0.0/lib -lmpdec -lm

@erlend-aasland

erlend-aasland commented May 3, 2024

Copy link
Copy Markdown
Contributor Author

@zware: I'm not sure if we should fall back to the internal copy if libmpdec is missing. For now, it is marked as missing if it is missing :) We could change it so configure fails if --with-system-libmpdec=yes and no mpdecimal library was found.

@erlend-aasland

erlend-aasland commented May 3, 2024

Copy link
Copy Markdown
Contributor Author

Things to consider:

  • Ubuntu 20.04 provides libmpdecimal 2.4 (no pkgconfig support): apt install libmpdec2
  • Ubuntu 22.04 provides libmpdecimal 2.5 (no pkgconfig support): apt install libmpdec3 (!)
  • Ubuntu 24.04 does not appear to provide libmpdecimal at all

For now, we can consider to use the bundled version for the Ubuntu CI.

@zware

zware commented May 3, 2024

Copy link
Copy Markdown
Member

@zware: I'm not sure if we should fall back to the internal copy if libmpdec is missing. For now, it is marked as missing if it is missing :) We could change it so configure fails if --with-system-libmpdec=yes and no mpdecimal library was found.

Right, an explicit --with-system-libmpdec argument should be followed, failing if the system library is not there. --with-system-libmpdec=no/--without-system-libmpdec should warn, and no argument with no system library available should also warn. Whatever I said in the issue was really shorthand for "do whatever we did when we ripped out libffi because that seemed to work1" which I think is basically what I've now laid out here :)

Footnotes

  1. though that one was complicated by differing timelines on Linux and macOS which we don't need to deal with here ↩

@erlend-aasland

erlend-aasland commented May 5, 2024

Copy link
Copy Markdown
Contributor Author

Right, an explicit --with-system-libmpdec argument should be followed, failing if the system library is not there.

Yep

[...] --with-system-libmpdec=no/--without-system-libmpdec should warn, [..]

Yes, we need to warn for that case.

[...] and no argument with no system library available should also warn.

Hm, I think this case should fail, similar to how we handle --with-system-libmpdec=yes and no library found.

@zware

zware commented May 5, 2024

Copy link
Copy Markdown
Member

[...] and no argument with no system library available should also warn.

Hm, I think this case should fail, similar to how we handle --with-system-libmpdec=yes and no library found.

I don't want a bare ./configure to start failing if mpdecimal isn't found, since that's not going to be the behavior even after the bundled library is removed. I could maybe get behind allowing the _decimal module build to fail rather than build the bundled library in that case, but even then we should also emit a warning from configure to make the issue more obvious.

@erlend-aasland

erlend-aasland commented May 5, 2024

Copy link
Copy Markdown
Contributor Author

Ok, how about this:

  • bare (no --with-system-libmpdec): warn if system library is not found, mark _decimal as missing
  • --with-system-libmpdec=yes: mark _decimal as missing
  • --with-system-libmpdec=no: use bundled version, warn that it will be removed in Python 3.15

@erlend-aasland erlend-aasland marked this pull request as ready for review May 5, 2024 22:48
@erlend-aasland erlend-aasland requested a review from zware May 5, 2024 22:48
zware
zware approved these changes May 6, 2024
View reviewed changes

@zware zware left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hide comment

I have a couple of questions and qualms, but overall I'm okay with this as is. We can tweak it through the beta phase if needed.

erlend-aasland and others added 2 commits May 6, 2024 09:02
@erlend-aasland @zware
Address review: add deprecated-removed directive regarding the remova… …
e3f3e17 
…l of the mpdecimal sources

Co-authored-by: Zachary Ware <zachary.ware@gmail.com>
@erlend-aasland
Add libmpdec-dev to CI deps, remove CI hack, and enforce mpdecimal ve… …
c84f2b7 
…rsion check in configure
15 hidden items Load more…
@erlend-aasland erlend-aasland marked this pull request as draft May 6, 2024 08:51
@erlend-aasland erlend-aasland marked this pull request as ready for review May 6, 2024 09:12
@erlend-aasland erlend-aasland requested a review from zware May 6, 2024 09:14
@erlend-aasland erlend-aasland merged commit 325a1da into python:main May 6, 2024
@erlend-aasland erlend-aasland deleted the libmpdec-default-to-system-lib branch May 6, 2024 19:16
@erlend-aasland

erlend-aasland commented May 6, 2024

Copy link
Copy Markdown
Contributor Author

Thanks for helping out, Zach!

SonicField pushed a commit to SonicField/cpython that referenced this pull request May 8, 2024
@erlend-aasland @zware @SonicField
pythongh-115119: Default to --with-system-libmpdec=yes (python#118539) …
b62d61f 
Co-authored-by: Zachary Ware <zachary.ware@gmail.com>
@asottile

asottile commented May 11, 2024

Copy link
Copy Markdown
Contributor

fwiw this is going to be painful for ubuntu and debian which has removed the libmpdec system library entirely (as it was only used for building python). also this was done pretty close to the beta freeze so it definitely caught me off guard (and broke the deadsnakes builds!)

@thesamesam

thesamesam commented May 11, 2024 •
edited
Loading

Copy link
Copy Markdown
Contributor

That seems to have been mentioned at #118539 (comment) already, but I'm not sure what the problem is. We didn't have libmpdec packaged in Gentoo until now but we've done it and now all is fine...

Why can't Debian/Ubuntu just set --with-system-libmpdec=no or --without-system-libmpdec (whichever it is) if they can't package libmpdec quickly?

@ned-deily ned-deily mentioned this pull request May 19, 2024
Merged
@erlend-aasland

erlend-aasland commented May 20, 2024

Copy link
Copy Markdown
Contributor Author

@thesamesam, @asottile: we're adding a fallback to the bundled version if an external libmpdec cannot be found.

FTR, I find the recent removal of libmpdec from Ubuntu/Debian very strange.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zware zware zware approved these changes

@corona10 corona10 Awaiting requested review from corona10

@ezio-melotti ezio-melotti Awaiting requested review from ezio-melotti ezio-melotti is a code owner

@hugovk hugovk hugovk is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@erlend-aasland @zware @asottile @thesamesam