beir evaluation quirks

**Dataset(s)**

- [beir/arguana](https://ir-datasets.com/beir.html#beir/arguana)
- [beir/quora](https://ir-datasets.com/beir.html#beir/quora) /dev /test

**Describe the proposed change**

This twitter thread points out some evaluation quirks when using these datasets: https://twitter.com/nandan__thakur/status/1603920955679551488

It's worth adding these details to the documentation page. Maybe it's important enough that it should also be logged when you load those datasets?