Proposal: Download.Read() should always return some bytes until EOF

[Erikvv]

The user usually calls Download.Read(p []byte) (n int, err error) in a loop. Currently at the boundary between two segments one call returns 0 bytes.

This causes bugs and additional complexity for library authors and users. Example:

storj-thirdparty/nextcloud-app#4 / storj-thirdparty/uplink-php#12
storj/uplink-java#10

I propose we change the behavior so that Download.Read() never returns 0 bytes, unless there are no more bytes left to download.

I'm not sure where this behaviour actually emerges and should be changed. Maybe somewhere in storj.io/common/ranger.concatReader or storj.io/common/readcloser.multiReadCloser.

[egonelbre]

An easy fix would be to retry in the top-level download implementation.

[egonelbre]

Oh, also, it could also be a side-effect of avoiding to return error and read 0 bytes from some internal read. (I'm not sure, but I think we had that property in https://github.com/storj/uplink/blob/main/private/piecestore/download.go#L114)

[ifraixedes]

@Erikvv does the uplink-c library return an error (specifically EOF) when the 0 bytes are returned at the boundary between 2 segments?

[Erikvv]

@ifraixedes No there's no error, EOF is only at the end of the object (or requested range).