"For Vista and later versions of Windows these warnings are presumably unnecessary since CancelIoEx() is used."
As close() on regular files, I would prefer to call explicitly cancel() to control exactly when the overlapped operation is cancelled. Can't you fix multiprocessing and/or the unit test to ensure that all overlapped operations are completed or cancelled?
close() does flush buffers and so may fail. Is it the same for CancelIo/CancelIoEx? In the official documentation, only one error case is described: "If this function cannot find a request to cancel, the return value is 0 (zero), and GetLastError returns ERROR_NOT_FOUND."
The warning is useful because it may be a real bug in the code. I also like ResourceWarning("unclosed file/socket ...") warnings.