In the current implementation of connect/accept, it is possible for
`accept` to complete (i.e., return a non-NULL communicator) after the
corresponding `connect` returned a NULL communicator (while waiting for
a completion for the connection message). This is a strange semantic,
and evidently causes NCCL to be unhappy, particularly in the multi-recv
case (which is being added in a future commit).

So, after sending the connect message, defer waiting for completion;
block when closing the send comm if necessary.

Signed-off-by: Eric Raut <[email protected]>

These tests do not support multi-recv.

Signed-off-by: Eric Raut <[email protected]>

The previous implementation of the ring unit test posts sends before
receives, which is incompatible with the current multi-recv behavior of
rejecting sends until ctrl information is avaiable.

Signed-off-by: Eric Raut <[email protected]>

msgbuff functions accept tag and multi-recv information. RDMA protocol
code is updated to pass dummy values for these fields. Multi-recv
support for RDMA protocol will be an upcoming commit.

Signed-off-by: Eric Raut <[email protected]>

The multi-recv interface allows aggregating up to 8 receive requests in
a single request.

This commit does not yet advertise support for multi-recv to NCCL.

* Temporarily disables eager; it will be re-enabled in a future commit.

Signed-off-by: Eric Raut <[email protected]>

RDMA protocol will now support up to 8 multi-recv buffers at a time.

Signed-off-by: Eric Raut <[email protected]>