> Not exactly. The select is done on the queue's pipe and on the workers'
> fds *at the same time*. Thus there's no race condition.
You're right, I missed this part, it's perfectly safe.
But I think there's a problem with the new implementation in Python.
Writes to a pipe are guaranteed to be atomic if you write less than
PIPE_BUF (4K on Linux, 512 by POSIX) at a time. Writes to a datagram
Unix domain socket are also atomic.
But Lib/multiprocessing/connection.py does:
def _send_bytes(self, buf):
# For wire compatibility with 3.2 and lower
n = len(buf)
self._send(struct.pack("=i", len(buf)))
# The condition is necessary to avoid "broken pipe" errors
# when sending a 0-length buffer if the other end closed the pipe.
if n > 0:
self._send(buf)
This is definitely not atomic. If two processes write objects of
different size at the same time, it can probably lead to trouble.
Also, Pipe(duplex=True) should probably return a SOCK_DGRAM Unix
socket for the same reason.
If I missed something here, I promise to shut up ;-)