issues with default block_size

Thanks for the wonderful library!

I have an issue using data from a remote http url, here is a example snippet ( originally, the issue arose while using the dataset in an intake catalog ) :

fs_of = fs.open_files('https://zenodo.org/record/3819896/files/Av.fesom.1948.nc?download=1')
of = fs_of[0].open()
xr.open_dataset(of)
......
~/miniconda3/envs/pyfesom2/lib/python3.7/site-packages/fsspec/caching.py in _fetch(self, start, end)
    337         ):
    338             # First read, or extending both before and after
--> 339             self.cache = self.fetcher(start, bend)
    340             self.start = start
    341         elif start < self.start:

~/miniconda3/envs/pyfesom2/lib/python3.7/site-packages/fsspec/implementations/http.py in _fetch_range(self, start, end)
    317             else:
    318                 raise ValueError(
--> 319                     "Got more bytes (%i) than requested (%i)" % (cl, end - start)
    320                 )
    321         else:
ValueError: Got more bytes (13470624) than requested (5242888)

with a little digging i figured that fspec made range-request in http.py while server sent entire file and the response was larger then block_size.

Here is response header from the server:

!curl -I "https://zenodo.org/record/3819896/files/Av.fesom.1948.nc?download=1"

HTTP/1.1 200 OK
Server: nginx/1.16.1
Content-Type: application/octet-stream
Content-Length: 13470624
Content-MD5: 008f40a425030d78cc7de2b2ecec12be
Content-Disposition: attachment; filename=Av.fesom.1948.nc
.....
ETag: "md5:008f40a425030d78cc7de2b2ecec12be"
Last-Modified: Mon, 11 May 2020 09:23:33 GMT
Date: Thu, 21 May 2020 21:40:03 GMT
Accept-Ranges: none
X-RateLimit-Limit: 60
X-RateLimit-Remaining: 59
X-RateLimit-Reset: 1590097264
Retry-After: 60
....

when server says Accept-Range:none or in some other cases when it doesn't exist, shouldn't we by default do a raw download or fetchall instead of attempting range requests by default? I think this would add default support to lot of datasets on servers( outdated?) like zenodo etc.

it does work when using block_size = filesize or higher like:

fs_of=fs.open_files('https://zenodo.org/record/3819896/files/Av.fesom.1948.nc?download=1', block_size=13470624) # or some high value
of = fs_of[0].open()
xr.open_dataset(of)

, but that is a little burden on the user. For some reason block_size=0 did not work, i am not sure if that is from xarray or fsspec, so leaving that out for now.

Start by reproducing the Zenodo URL with the HTTP implementation in fsspec/implementations/http.py and inspect the cache behavior shown in fsspec/caching.py. Done should define and verify behavior for servers reporting Accept-Ranges: none or omitting it, including the reported default block_size and block_size=0 cases.