Alright, it bothered me so I wrote a small C testcase (attached),
which calls malloc in a loop, and can call memset upon the allocated
block right after allocation:
$ gcc -o /tmp/test /tmp/test.c; /tmp/test
malloc() returned NULL after 3050MB
$ gcc -DDO_MEMSET -o /tmp/test /tmp/test.c; /tmp/test
malloc() returned NULL after 2130MB
Without memset, the kernel happily allocates until we reach the 3GB
user address space limit.
With memset, it bails out way before.
I don't know what this'll give on 64-bit, but I assume one should get
comparable result.
I would guess that the reason why the Python list allocation fails is
because of the exponential allocation scheme: since memory is
allocated in large chunks before being used, the kernel happily
overallocates.
With a more progressive allocation+usage, it should return ENOMEM at some point.
Anyway, that's probably off-topic!