OverflowError: There was an overflow in the <class 'pyarrow.lib.ListArray'>. Try to reduce writer_batch_size to have batches smaller than 2GB

## Describe the bug
Despite having batches way under 2Gb when running `datasets.map()`, after processing correctly the data of the first batch without fuss and irrespective of writer_batch_size (say 2,4,8,16,32,64 and 128 in my case), it returns the following error :

> OverflowError: There was an overflow in the . Try to reduce writer_batch_size to have batches smaller than 2GB

Note that I always run `batch_size=writer_batch_size` :

## Steps to reproduce the bug
```python
datasets.map(lambda example : {"column_name" : function(arguments)}, batched=False, remove_columns = datasets.column_names, batch_size=batch_size, writer_batch_size=batch_size, disable_nullable=True, num_proc=None, desc="blablabla")
```
## Introspecting CUDA memory during bug

Placed within `function(arguments)` the following statement to introspect memory usage, merely a little over 1/4 of 2Gb
`print(torch.cuda.memory_summary(device=device, abbreviated=False))`

> |===========================================================================|
| PyTorch CUDA memory summary, device ID 0 |
|---------------------------------------------------------------------------|
| CUDA OOMs: 0 | cudaMalloc retries: 0 |
|===========================================================================|
| Metric | Cur Usage | Peak Usage | Tot Alloc | Tot Freed |
|---------------------------------------------------------------------------|
| Allocated memory | 541418 KB | 545725 KB | 555695 KB | 14276 KB |
| from large pool | 540672 KB | 544431 KB | 544431 KB | 3759 KB |
| from small pool | 746 KB | 1714 KB | 11264 KB | 10517 KB |
|---------------------------------------------------------------------------|
| Active memory | 541418 KB | 545725 KB | 555695 KB | 14276 KB |
| from large pool | 540672 KB | 544431 KB | 544431 KB | 3759 KB |
| from small pool | 746 KB | 1714 KB | 11264 KB | 10517 KB |
|---------------------------------------------------------------------------|
| GPU reserved memory | 598016 KB | 598016 KB | 598016 KB | 0 B |
| from large pool | 595968 KB | 595968 KB | 595968 KB | 0 B |
| from small pool | 2048 KB | 2048 KB | 2048 KB | 0 B |
|---------------------------------------------------------------------------|
| Non-releasable memory | 36117 KB | 52292 KB | 274275 KB | 238158 KB |
| from large pool | 34816 KB | 51537 KB | 261713 KB | 226897 KB |
| from small pool | 1301 KB | 2045 KB | 12562 KB | 11261 KB |
|---------------------------------------------------------------------------|
| Allocations | 198 | 224 | 478 | 280 |
| from large pool | 74 | 75 | 75 | 1 |
| from small pool | 124 | 150 | 403 | 279 |
|---------------------------------------------------------------------------|
| Active allocs | 198 | 224 | 478 | 280 |
| from large pool | 74 | 75 | 75 | 1 |
| from small pool | 124 | 150 | 403 | 279 |
|---------------------------------------------------------------------------|
| GPU reserved segments | 21 | 21 | 21 | 0 |
| from large pool | 20 | 20 | 20 | 0 |
| from small pool | 1 | 1 | 1 | 0 |
|---------------------------------------------------------------------------|
| Non-releasable allocs | 18 | 23 | 166 | 148 |
| from large pool | 17 | 18 | 19 | 2 |
| from small pool | 1 | 6 | 147 | 146 |
|===========================================================================|

## Expected results
Efficiently process the datasets and write it down to disk.

## Actual results
--------------------------------------------------------------------------
OverflowError Traceback (most recent call last)
~\anaconda3\envs\xxx\lib\site-packages\datasets\arrow_dataset.py in _map_single(self, function, with_indices, input_columns, batched, batch_size, drop_last_batch, remove_columns, keep_in_memory, load_from_cache_file, cache_file_name, writer_batch_size, features, disable_nullable, fn_kwargs, new_fingerprint, rank, offset, disable_tqdm, desc, cache_only)
2390 else:
-> 2391 writer.write(example)
2392 else:

~\anaconda3\envs\xxx\lib\site-packages\datasets\arrow_writer.py in write(self, example, key, writer_batch_size)
367
--> 368 self.write_examples_on_file()
369

~\anaconda3\envs\xxx\lib\site-packages\datasets\arrow_writer.py in write_examples_on_file(self)
316 if not isinstance(pa_array[0], pa.lib.FloatScalar):
--> 317 raise OverflowError(
318 "There was an overflow in the {}. Try to reduce writer_batch_size to have batches smaller than 2GB".format(

OverflowError: There was an overflow in the . Try to reduce writer_batch_size to have batches smaller than 2GB

During handling of the above exception, another exception occurred:

OverflowError Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_16268/2456940807.py in
3 #tracker = OfflineEmissionsTracker(country_iso_code="FRA", project_name='xxx'+time_stamp,output_dir='./codecarbon')
4 #tracker.start()
----> 5 process_datasets(source_datasets_paths, dataset_dir, LM_tokenizer, LMhead_model, datasets_selection=['wikipedia'], from_scratch=True,
6 clean_sentences=False, negative_sampling=False, translate=False, tokenize=False, generate_embeddings=True, concatenate_embeddings=False,
7 max_sample=10000, padding='do_not_pad', truncation=True, cpu_batch_size=1000, gpu_batch_size=2, cpu_writer_batch_size=1000, gpu_writer_batch_size=2, disable_nullable=True, num_proc=None) #

~\xxx\xxx.py in process_datasets(source_datasets_paths, dataset_dir, LM_tokenizer, LMhead_model, datasets_selection, from_scratch, clean_sentences, translate, negative_sampling, tokenize, generate_embeddings, concatenate_embeddings, max_sample, padding, truncation, cpu_batch_size, gpu_batch_size, cpu_writer_batch_size, gpu_writer_batch_size, disable_nullable, num_proc)
481 for column in tqdm(dataset.column_names, desc=f'Processing column', leave=False):
482 if "xxx_" in column:
--> 483 dataset = dataset.map(lambda example :
484 {"embeddings_"+str(column).replace("translated_",""):function(input_ids=example[column],
485 token_type_ids=example[column.replace("input_ids","token_type_ids")],

~\anaconda3\envs\xxx\lib\site-packages\datasets\arrow_dataset.py in map(self, function, with_indices, input_columns, batched, batch_size, drop_last_batch, remove_columns, keep_in_memory, load_from_cache_file, cache_file_name, writer_batch_size, features, disable_nullable, fn_kwargs, num_proc, suffix_template, new_fingerprint, desc)
2034
2035 if num_proc is None or num_proc == 1:
-> 2036 return self._map_single(
2037 function=function,
2038 with_indices=with_indices,

~\anaconda3\envs\xxx\lib\site-packages\datasets\arrow_dataset.py in wrapper(*args, **kwargs)
501 self: "Dataset" = kwargs.pop("self")
502 # apply actual function
--> 503 out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
504 datasets: List["Dataset"] = list(out.values()) if isinstance(out, dict) else [out]
505 for dataset in datasets:

~\anaconda3\envs\xxx\lib\site-packages\datasets\arrow_dataset.py in wrapper(*args, **kwargs)
468 }
469 # apply actual function
--> 470 out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
471 datasets: List["Dataset"] = list(out.values()) if isinstance(out, dict) else [out]
472 # re-apply format to the output

~\anaconda3\envs\xxx\lib\site-packages\datasets\fingerprint.py in wrapper(*args, **kwargs)
404 # Call actual function
405
--> 406 out = func(self, *args, **kwargs)
407
408 # Update fingerprint of in-place transforms + update in-place history of transforms

~\anaconda3\envs\xxx\lib\site-packages\datasets\arrow_dataset.py in _map_single(self, function, with_indices, input_columns, batched, batch_size, drop_last_batch, remove_columns, keep_in_memory, load_from_cache_file, cache_file_name, writer_batch_size, features, disable_nullable, fn_kwargs, new_fingerprint, rank, offset, disable_tqdm, desc, cache_only)
2425 if update_data:
2426 if writer is not None:
-> 2427 writer.finalize()
2428 if tmp_file is not None:
2429 tmp_file.close()

~\anaconda3\envs\xxx\lib\site-packages\datasets\arrow_writer.py in finalize(self, close_stream)
440 # Re-intializing to empty list for next batch
441 self.hkey_record = []
--> 442 self.write_examples_on_file()
443 if self.pa_writer is None:
444 if self._schema is not None:

~\anaconda3\envs\xxx\lib\site-packages\datasets\arrow_writer.py in write_examples_on_file(self)
315 # This check fails with FloatArrays with nans, which is not what we want, so account for that:
316 if not isinstance(pa_array[0], pa.lib.FloatScalar):
--> 317 raise OverflowError(
318 "There was an overflow in the {}. Try to reduce writer_batch_size to have batches smaller than 2GB".format(
319 type(pa_array)

OverflowError: There was an overflow in the . Try to reduce writer_batch_size to have batches smaller than 2GB

## Environment info

- `datasets` version: 1.13.3
- Platform: Windows-10-10.0.19042-SP0
- Python version: 3.8.11
- PyArrow version: 3.0.0

##Next steps
Testing on Linux.
@albertvillanova