◐ Shell
reader mode source ↗
Skip to content

bpo-29782: Consolidate _Py_Bit_Length()#20739

Merged
vstinner merged 21 commits into
python:masterfrom
niklasf:bpo-29782-msb
Jun 15, 2020
Merged

bpo-29782: Consolidate _Py_Bit_Length()#20739
vstinner merged 21 commits into
python:masterfrom
niklasf:bpo-29782-msb

Conversation

@niklasf

@niklasf niklasf commented Jun 8, 2020

Copy link
Copy Markdown
Contributor

In GH-2866, _Py_Bit_Length() was added to pymath.h for lack of a better
location. GH-20518 added a more appropriate header file for bit utilities. It
also shows how to properly use intrinsics. This allows reconsidering bpo-29782.

  • Move the function to the new header.
  • Changed return type to match __builtin_clzl and reviewed usage.
  • Use intrinsics where available.
  • Pick a (mostly theoretical) fallback implementation suitable for
    inlining.

https://bugs.python.org/issue29782

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hide comment

Maybe add test_bit_length() to _testinternalcapi.c: see test_popcount().

@vstinner

vstinner commented Jun 8, 2020

Copy link
Copy Markdown
Member

Previous rejected attempt: PR #594.

@vstinner

vstinner commented Jun 8, 2020

Copy link
Copy Markdown
Member

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hide comment

I disliked PR #594 because it added #include <intrin.h> to a new *public header file: Include/pyintrinsics.h. This PR is different, all changes are made in the internal C API. We have way more freedom in the internal C API.

@vstinner

vstinner commented Jun 8, 2020

Copy link
Copy Markdown
Member

The Windows build looks broken:

Run .\python.bat -m test.pythoninfo
Running Release|x64 interpreter...
##[error]Process completed with exit code 1.

@mdickinson mdickinson self-requested a review June 8, 2020 18:26
niklasf added 4 commits June 8, 2020 21:18
In pythonGH-2866, _Py_Bit_Length() was added to pymath.h for lack of a better
location. pythonGH-20518 added a more appropriate header file for bit utilities. It
also shows how to properly use intrinsics. This allows reconsidering bpo-29782.

* Move the function to the new header.
* Changed return type to match __builtin_clzl and reviewed usage.
* Use intrinsics where available.
* Pick a (mostly theoretical) fallback implementation suitable for
  inlining.
niklasf and others added 2 commits June 8, 2020 22:03
Co-authored-by: Victor Stinner <vstinner@python.org>
14 hidden items Load more…
@niklasf

niklasf commented Jun 8, 2020

Copy link
Copy Markdown
Contributor Author

Thanks for reviewing.

Also added unit tests as suggested (in addition to the existing test coverage). I briefly disabled the branch with clz (8e3ce66) to get a CI run with the fallback implementation. The current fallback is identical to the original (except for inlining).

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hide comment

Thanks for the many updates! The code now looks better! New review.

@vstinner

vstinner commented Jun 8, 2020

Copy link
Copy Markdown
Member

Using gcc 10 -O3 -flto, I get the following machine code in test_bit_length():

# input = rax
# if (x != 0)
test   rax,rax
je     0x7fffeaa99978 <test_bit_length+504>

# return (int)sizeof(unsigned long) * 8 - __builtin_clzl(x);
# return 0x40 - bsr(rax) ^ 0x3f
bsr    rax,rax
mov    ecx,0x40
xor    rax,0x3f
mov    r9d,ecx
sub    r9d,eax
# result = r9d

x86 BSR (Bit Scan Reverse) instruction is used as expected.

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hide comment

LGTM!

@mdickinson: Do you want to review this PR as well?

@mdickinson

Copy link
Copy Markdown
Member

@mdickinson: Do you want to review this PR as well?

I'd very much like to take a look, but am short on available time. Can you give me until the weekend? If I've failed to review by the start of next week, go ahead and merge.

@vstinner

vstinner commented Jun 9, 2020

Copy link
Copy Markdown
Member

@mdickinson: this PR can wait one week :-)

@mdickinson mdickinson left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hide comment

LGTM!

@vstinner vstinner merged commit 794e7d1 into python:master Jun 15, 2020
@vstinner

Copy link
Copy Markdown
Member

Thanks @niklasf, I merged your PR.

I removed "(mostly theoretical)" from your commit message:

* Pick a (mostly theoretical) fallback implementation suitable for
  inlining.

We support other C compilers, like XLC.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants