huggingface / huggingface/evaluate

Optionally stop data deletion after compute

Open
#641 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.5k
Forks
341
PR merge metrics
No merged PRs in 30d

Description

In the following locs I'm seeing that the module is deleting the data with `del self.data` can we have a flag that avoids this deletion so that we can drill down into good/bad data points at a later stage? I can raise a raise a PR if you think this is a good idea.

https://github.com/huggingface/evaluate/blob/55f1bc6e072b05c2d9db1589a07e20f38902b1ec/src/evaluate/module.py#L471

```python
...
if self.buf_writer is not None:
self.buf_writer = None
del self.data
self.data = None
else:
# Release locks and delete all the cache files. Process 0 is released last.
for filelock, file_path in reversed(list(zip(self.filelocks, self.file_paths))):
logger.info(f"Removing {file_path}")
del self.data
self.data = None
del self.writer
self.writer = None
os.remove(file_path)
filelock.release()
...
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.