bpo-26415: reduce peak memory consumption by the parser by tyomitch · Pull Request #10995 · python/cpython

the-knights-who-say-ni
Pyperformance (0.7.0) reports significant improvement of memory footprint, ranging from "1.03x smaller" to "1.44x smaller", for every benchmark apart from sympy_integrate:
### 2to3 ###
Mean +- std dev: 8597.4 kB +- 32.6 kB -> 6969.4 kB +- 13.3 kB: 1.23x smaller
Significant (t=206.57)

### chaos ###
Mean +- std dev: 9543.2 kB +- 60.5 kB -> 7886.2 kB +- 40.5 kB: 1.21x smaller
Significant (t=101.74)

### crypto_pyaes ###
Mean +- std dev: 9078.2 kB +- 83.4 kB -> 7434.6 kB +- 55.5 kB: 1.22x smaller
Significant (t=73.36)

### deltablue ###
Mean +- std dev: 9219.6 kB +- 45.4 kB -> 8025.4 kB +- 15.8 kB: 1.15x smaller
Significant (t=111.01)

### django_template ###
Mean +- std dev: 20.3 MB +- 97.8 kB -> 18.5 MB +- 32.3 kB: 1.10x smaller
Significant (t=79.60)

### dulwich_log ###
Mean +- std dev: 12.7 MB +- 71.5 kB -> 11.2 MB +- 29.9 kB: 1.14x smaller
Significant (t=90.93)

### fannkuch ###
Mean +- std dev: 8574.4 kB +- 27.8 kB -> 6969.8 kB +- 18.1 kB: 1.23x smaller
Significant (t=216.07)

### float ###
Mean +- std dev: 22.1 MB +- 44.8 kB -> 20.5 MB +- 25.8 kB: 1.08x smaller
Significant (t=138.63)

### go ###
Mean +- std dev: 10.2 MB +- 36.1 kB -> 8995.6 kB +- 39.7 kB: 1.16x smaller
Significant (t=122.30)

### hexiom ###
Mean +- std dev: 8778.2 kB +- 22.0 kB -> 7248.4 kB +- 21.5 kB: 1.21x smaller
Significant (t=222.50)

### html5lib ###
Mean +- std dev: 21.5 MB +- 67.3 kB -> 19.8 MB +- 37.5 kB: 1.09x smaller
Significant (t=104.50)

### json_dumps ###
Mean +- std dev: 9760.8 kB +- 56.1 kB -> 8188.2 kB +- 42.4 kB: 1.19x smaller
Significant (t=100.05)

### json_loads ###
Mean +- std dev: 9190.0 kB +- 43.3 kB -> 7607.4 kB +- 23.3 kB: 1.21x smaller
Significant (t=143.88)

### logging_format ###
Mean +- std dev: 9194.6 kB +- 34.0 kB -> 8670.2 kB +- 199.4 kB: 1.06x smaller
Significant (t=11.59)

### logging_silent ###
Mean +- std dev: 8906.8 kB +- 36.9 kB -> 7312.2 kB +- 23.8 kB: 1.22x smaller
Significant (t=162.35)

### logging_simple ###
Mean +- std dev: 9131.8 kB +- 40.6 kB -> 8290.8 kB +- 103.1 kB: 1.10x smaller
Significant (t=33.96)

### mako ###
Mean +- std dev: 16.4 MB +- 583.3 kB -> 14.9 MB +- 416.7 kB: 1.10x smaller
Significant (t=9.27)

### meteor_contest ###
Mean +- std dev: 10.5 MB +- 46.2 kB -> 9184.0 kB +- 37.4 kB: 1.17x smaller
Significant (t=115.84)

### nbody ###
Mean +- std dev: 8653.8 kB +- 61.6 kB -> 7232.4 kB +- 45.7 kB: 1.20x smaller
Significant (t=82.90)

### nqueens ###
Mean +- std dev: 8828.2 kB +- 42.8 kB -> 7417.8 kB +- 52.4 kB: 1.19x smaller
Significant (t=93.29)

### pathlib ###
Mean +- std dev: 10.3 MB +- 84.6 kB -> 9027.6 kB +- 68.3 kB: 1.17x smaller
Significant (t=61.66)

### pickle ###
Mean +- std dev: 9473.2 kB +- 64.0 kB -> 7970.4 kB +- 74.5 kB: 1.19x smaller
Significant (t=68.39)

### pickle_dict ###
Mean +- std dev: 9550.6 kB +- 107.8 kB -> 7859.0 kB +- 70.4 kB: 1.22x smaller
Significant (t=58.76)

### pickle_list ###
Mean +- std dev: 9490.6 kB +- 54.5 kB -> 7799.0 kB +- 25.8 kB: 1.22x smaller
Significant (t=125.48)

### pickle_pure_python ###
Mean +- std dev: 9429.2 kB +- 73.2 kB -> 7836.8 kB +- 19.6 kB: 1.20x smaller
Significant (t=94.00)

### pidigits ###
Mean +- std dev: 8747.4 kB +- 50.3 kB -> 7234.8 kB +- 43.2 kB: 1.21x smaller
Significant (t=102.04)

### python_startup ###
Mean +- std dev: 10.5 MB +- 22.7 kB -> 7604.2 kB +- 26.4 kB: 1.42x smaller
Significant (t=407.98)

### python_startup_no_site ###
Mean +- std dev: 10.0 MB +- 19.0 kB -> 7106.6 kB +- 29.3 kB: 1.44x smaller
Significant (t=404.06)

### raytrace ###
Mean +- std dev: 9054.4 kB +- 61.1 kB -> 7533.4 kB +- 86.7 kB: 1.20x smaller
Significant (t=64.12)

### regex_compile ###
Mean +- std dev: 9840.8 kB +- 23.3 kB -> 8361.4 kB +- 54.5 kB: 1.18x smaller
Significant (t=111.65)

### regex_dna ###
Mean +- std dev: 14.7 MB +- 51.6 kB -> 13.2 MB +- 72.1 kB: 1.11x smaller
Significant (t=74.04)

### regex_effbot ###
Mean +- std dev: 9273.6 kB +- 21.1 kB -> 7749.8 kB +- 28.3 kB: 1.20x smaller
Significant (t=193.05)

### regex_v8 ###
Mean +- std dev: 9785.2 kB +- 54.8 kB -> 8162.8 kB +- 39.7 kB: 1.20x smaller
Significant (t=107.14)

### richards ###
Mean +- std dev: 8731.4 kB +- 43.6 kB -> 7262.2 kB +- 56.1 kB: 1.20x smaller
Significant (t=92.50)

### scimark_fft ###
Mean +- std dev: 8784.6 kB +- 19.9 kB -> 7247.8 kB +- 34.4 kB: 1.21x smaller
Significant (t=173.03)

### scimark_lu ###
Mean +- std dev: 8807.6 kB +- 23.2 kB -> 7222.8 kB +- 34.2 kB: 1.22x smaller
Significant (t=171.59)

### scimark_monte_carlo ###
Mean +- std dev: 8745.2 kB +- 21.1 kB -> 7147.2 kB +- 32.5 kB: 1.22x smaller
Significant (t=184.62)

### scimark_sor ###
Mean +- std dev: 8726.8 kB +- 16.0 kB -> 7151.6 kB +- 30.5 kB: 1.22x smaller
Significant (t=204.58)

### scimark_sparse_mat_mult ###
Mean +- std dev: 9240.2 kB +- 30.5 kB -> 7652.8 kB +- 29.3 kB: 1.21x smaller
Significant (t=168.01)

### spectral_norm ###
Mean +- std dev: 8583.4 kB +- 27.5 kB -> 7090.2 kB +- 39.4 kB: 1.21x smaller
Significant (t=139.03)

### sqlalchemy_declarative ###
Mean +- std dev: 19.5 MB +- 34.6 kB -> 17.9 MB +- 21.6 kB: 1.09x smaller
Significant (t=177.78)

### sqlalchemy_imperative ###
Mean +- std dev: 18.9 MB +- 67.8 kB -> 17.2 MB +- 35.5 kB: 1.10x smaller
Significant (t=100.06)

### sqlite_synth ###
Mean +- std dev: 8818.4 kB +- 42.1 kB -> 7376.6 kB +- 15.4 kB: 1.20x smaller
Significant (t=143.69)

### sympy_expand ###
Mean +- std dev: 33.0 MB +- 60.3 kB -> 31.5 MB +- 33.3 kB: 1.05x smaller
Significant (t=103.54)

### sympy_integrate ###
Mean +- std dev: 32.2 MB +- 47.6 kB -> 31.6 MB +- 61.4 kB: 1.02x smaller
Not significant

### sympy_str ###
Mean +- std dev: 33.8 MB +- 43.1 kB -> 32.2 MB +- 68.0 kB: 1.05x smaller
Significant (t=91.21)

### sympy_sum ###
Mean +- std dev: 57.0 MB +- 58.1 kB -> 55.4 MB +- 63.6 kB: 1.03x smaller
Significant (t=86.36)

### telco ###
Mean +- std dev: 9172.2 kB +- 30.2 kB -> 7515.8 kB +- 68.0 kB: 1.22x smaller
Significant (t=99.61)

### tornado_http ###
Mean +- std dev: 25.7 MB +- 267.7 kB -> 24.1 MB +- 102.6 kB: 1.07x smaller
Significant (t=25.77)

### unpack_sequence ###
Mean +- std dev: 8913.4 kB +- 38.7 kB -> 7315.2 kB +- 13.1 kB: 1.22x smaller
Significant (t=174.87)

### unpickle ###
Mean +- std dev: 9483.8 kB +- 47.4 kB -> 7769.6 kB +- 23.2 kB: 1.22x smaller
Significant (t=145.30)

### unpickle_list ###
Mean +- std dev: 9474.8 kB +- 55.5 kB -> 7773.4 kB +- 27.2 kB: 1.22x smaller
Significant (t=123.16)

### unpickle_pure_python ###
Mean +- std dev: 9489.4 kB +- 92.1 kB -> 7857.8 kB +- 27.8 kB: 1.21x smaller
Significant (t=75.81)

### xml_etree_generate ###
Mean +- std dev: 14.1 MB +- 85.2 kB -> 12.2 MB +- 123.0 kB: 1.15x smaller
Significant (t=56.98)

### xml_etree_iterparse ###
Mean +- std dev: 13.5 MB +- 30.3 kB -> 11.8 MB +- 87.5 kB: 1.15x smaller
Significant (t=85.90)

### xml_etree_parse ###
Mean +- std dev: 13.6 MB +- 60.6 kB -> 11.8 MB +- 66.8 kB: 1.15x smaller
Significant (t=91.00)

### xml_etree_process ###
Mean +- std dev: 14.2 MB +- 29.9 kB -> 12.5 MB +- 88.0 kB: 1.13x smaller
Significant (t=83.13)
Effects on benchmark times are less substantial, and range from "1.18x faster" for python_startup_no_site, to "1.22x slower" for unpack_sequence. The latter is likely pure noise because the measured times are in nanoseconds range.