Cache clean up might fail on Windows
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
Details can be found in this PR:
https://github.com/apache/beam/pull/12779
```
apache_beam\runners\interactive\recording_manager_test.py:75:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam\runners\interactive\interactive_environment.py:118:
in new_env
_interactive_beam_env.cleanup()
apache_beam\runners\interactive\interactive_environment.py:272:
in cleanup
cache_manager.cleanup()
apache_beam\runners\interactive\caching\streaming_cache.py:391:
in cleanup
shutil.rmtree(self._cache_dir)
c:\hostedtoolcache\windows\python\3.6.8\x64\lib\shutil.py:500:
in rmtree
return _rmtree_unsafe(path, onerror)
c:\hostedtoolcache\windows\python\3.6.8\x64\lib\shutil.py:390:
in _rmtree_unsafe
_rmtree_unsafe(fullname, onerror)
c:\hostedtoolcache\windows\python\3.6.8\x64\lib\shutil.py:395:
in _rmtree_unsafe
onerror(os.unlink, fullname, sys.exc_info())
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
...
> os.unlink(fullname)
E
PermissionError: [WinError 32] The process cannot access the file because it is being
used by another process: 'D:\\a\\beam\\beam\\sdks\\python\\target\\.tox\\py36-win\\tmp\\it-4m1c1oje2145793178144\\full\\fb91a47796-2145832985040-2145832986608-2145793178144'
```
This does not happen on Linux-like systems:
https://user-images.githubusercontent.com/4423149/100034273-b343ef80-2db0-11eb-8988-dcfc2c79322c.png
A potential solution is to enable ignore_errors and log about manually cleaning up onerror.
Imported from Jira [BEAM-11339](https://issues.apache.org/jira/browse/BEAM-11339). Original Jira may contain additional context.
Reported by: ningk.
Contributor guide
Assessment
This issue has not been assessed yet.