GH-101291: Rearrange the size bits in PyLongObject#102464

markshannon

This PR rearranges the bits in what was ob_size, to slightly speedup the most common operations and to prepare for storing the tagged 2-complement value directly in a future PR.

The new layout is as follows:

Bits 0 and 1: 1 - sign. I.e. 0 for positive numbers, 1 for zero and 2 for negative numbers.
Bit 2 reserved (probably for the immortal bit)
Bits 3+ the unsigned size.

The bulk of the change is removing all the uses of Py_SIZE and Py_SETSIZE, and replacing them with a new set of inline functions.
It disturbs me how much we use unchecked casts, but that's a separate issue...

This will, inevitably, break Cython generated code again.

Performance measurement shows no significant change: https://github.com/faster-cpython/benchmarking/tree/main/results/bm-20230302-3.12.0a5%2B-ce6bfb2

Issue: Restore (or beat) Python 2 performance for arithmetic operations on ints that fit into a single word #101291

…tCount().

…Long_SignedDigitCount which might not be optimal, but is safe.

…header file.

…ion of immortal ints and tagged medium ints.

gvanrossum

As my flight might depart soon here's a first batch of review comments. Still to do longobject.c, and some modules.

gvanrossum

Here's the rest. I went over every diff chunk in longobject.c. Let's get this merged...

gvanrossum

Go for it!

sobolevn

I've opened #102940 to fix two new warnings from this PR :)

…2464) * Eliminate all remaining uses of Py_SIZE and Py_SET_SIZE on PyLongObject, adding asserts. * Change layout of size/sign bits in longobject to support future addition of immortal ints and tagged medium ints. * Add functions to hide some internals of long object, and for setting sign and digit count. * Replace uses of IS_MEDIUM_VALUE macro with _PyLong_IsCompact().

scoder

ISTM that the simple accessor functions like IsZero, IsPositive, IsNegative should be publicly available.

What about this part? Looks like it was dropped on the floor along the way.

) See python/cpython#102464

…2464) * Eliminate all remaining uses of Py_SIZE and Py_SET_SIZE on PyLongObject, adding asserts. * Change layout of size/sign bits in longobject to support future addition of immortal ints and tagged medium ints. * Add functions to hide some internals of long object, and for setting sign and digit count. * Replace uses of IS_MEDIUM_VALUE macro with _PyLong_IsCompact().

markshannon added 20 commits February 28, 2023 11:59

Add functions to hide some internals of long object.

0ec07e4

Add internal functions to longobject.c for setting sign and digit count.

292b9d0

Replace Py_SIZE(x) < 0 with _PyLong_IsNegative(x) in longobject.c

5c54894

Replace Py_ABS(Py_SIZE(a)) with _PyLong_DigitCount(a) in longobject.c

029aaa4

Remove many uses of Py_SIZE in longobject.c

b56e6da

Remove _PyLong_AssignValue, as it is no longer used.

91269fc

Remove some more uses of Py_SIZE in longobject.c.

c48e825

Remove a few more uses of Py_SIZE in longobject.c.

449c0e2

Remove some more uses of Py_SIZE, replacing with _PyLong_UnsignedDigi… …

c5ba601

…tCount().

Replace a few Py_SIZE() with _PyLong_SameSign().

4b3a3e8

Remove a few more Py_SIZE() from longobject.c

9ef9d2c

Replace uses of IS_MEDIUM_VALUE macro with _PyLong_IsSingleDigit.

9c408c1

Remove most of the remaining uses of Py_SIZE in longobject.c

548d656

Replace last remaining uses of Py_SIZE applied to longobject with _Py… …

3e3fefd

…Long_SignedDigitCount which might not be optimal, but is safe.

Don't use _PyObject_InitVar and move a couple of inline functions to … …

391fb51

…header file.

Correct name of inline function.

df8c7d3

Eliminate all remaining uses of Py_SIZE and Py_SET_SIZE on PyLongObject.

bc14fa6

Change layout of size/sign bits in longobject to support future addit… …

54c6f1b

…ion of immortal ints and tagged medium ints.

Test pairs of longs together on fast path of add/mul/sub.

ce6bfb2

Tidy up comment and delete commented out code.

4c1956b

markshannon requested review from isidentical, rhettinger and tiran as code owners March 6, 2023 09:57

bedevere-bot mentioned this pull request Mar 6, 2023

Restore (or beat) Python 2 performance for arithmetic operations on ints that fit into a single word #101291

Open

bedevere-bot added the awaiting core review label Mar 6, 2023

corona10 requested a review from mdickinson March 6, 2023 10:01

markshannon added 4 commits March 6, 2023 10:02

Add news.

301158b

Remove debugging asserts.

1aa1891

Fix storage classes.

bf2a9af

Remove development debug functions.

169f521

markshannon added 2 commits March 16, 2023 19:26

Replace _PyLong_Sign(x) < 0 with _PyLong_IsNegative(x).

f764aa8

fix sign check

9843ac0

gvanrossum reviewed Mar 17, 2023

View reviewed changes

markshannon mentioned this pull request Mar 18, 2023

gh-102509: Start initializing ob_digit of _PyLongValue #102510

Merged

lpereira reviewed Mar 21, 2023

View reviewed changes

gvanrossum reviewed Mar 21, 2023

View reviewed changes

markshannon added 2 commits March 22, 2023 11:23

Address some review comments.

d6cb917

Change asserts on digit counts to asserts on sign where applicable.

469d26f

gvanrossum approved these changes Mar 22, 2023

View reviewed changes

bedevere-bot added and removed awaiting core review labels Mar 22, 2023

markshannon merged commit 7559f5f into python:main Mar 22, 2023

bedevere-bot removed the awaiting merge label Mar 22, 2023

markshannon deleted the long-rearrange-size-bits branch March 22, 2023 14:50

scoder mentioned this pull request Apr 3, 2023

Implement support for the new PyLong struct layout in Py3.12a7 cython/cython#5353

Merged

gvanrossum mentioned this pull request Apr 3, 2023

gh-84436: Implement Immortal Objects #19474

Merged

scoder added a commit to cython/cython that referenced this pull request Apr 5, 2023

Implement support for the new PyLong struct layout in Py3.12a7. (GH-5353 …

781b087

) See python/cpython#102464

eduardo-elizondo reviewed Apr 7, 2023

View reviewed changes

skirpichev mentioned this pull request Apr 24, 2023

Breaking changes in Python 3.12.0a7 release gmpy2/gmpy2#405

Closed

verhovsky reviewed Apr 29, 2023

View reviewed changes

carljm mentioned this pull request May 5, 2023

build fails with --enable-pystats --with-pydebug (use of Py_SIZE on PyLongObject) #104184

Closed

eduardo-elizondo mentioned this pull request Jun 19, 2023

gh-84436: Add static flag in PyLongObject's lv_tag #103403

Closed

chris-eibl mentioned this pull request Dec 7, 2024

gh-127119: Faster check for small ints in long_dealloc #127620

Merged

eendebakpt mentioned this pull request Dec 7, 2024

gh-127119: Remove check on accidental deallocation of immortal objects for free-threaded build #127120

Closed

skirpichev mentioned this pull request Apr 25, 2025

sum() several times slower on Python 3 64-bit #68264

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

GH-101291: Rearrange the size bits in PyLongObject#102464

GH-101291: Rearrange the size bits in PyLongObject#102464
markshannon merged 37 commits into
python:mainfrom
faster-cpython:long-rearrange-size-bits

markshannon commented Mar 6, 2023 •

edited by gvanrossum

Loading

Uh oh!

gvanrossum left a comment

Uh oh!

gvanrossum left a comment

Uh oh!

gvanrossum left a comment

Uh oh!

sobolevn commented Mar 23, 2023

Uh oh!

scoder commented Mar 28, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

Conversation

markshannon commented Mar 6, 2023 • edited by gvanrossum Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gvanrossum left a comment

Choose a reason for hiding this comment

Uh oh!

gvanrossum left a comment

Choose a reason for hiding this comment

Uh oh!

gvanrossum left a comment

Choose a reason for hiding this comment

Uh oh!

sobolevn commented Mar 23, 2023

Uh oh!

scoder commented Mar 28, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

markshannon commented Mar 6, 2023 •

edited by gvanrossum

Loading