Local temp files are not always cleaned up
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 2d 55m
- Merged PRs (30d)
- 182
Description
Over time the temporary .tar.gz files accumulate on servers' local disk, requiring manual cleanup. This could be improved with a periodic task to ensure leftover files are removed.
Sample accumulation over a couple months:
```
[docker@<...>/dataDir/_REALTIME #]l
consumers/ __29__760__20230813T2316Z.tar.gz __5__1902__20230820T0323Z.tar.gz
__11__1180__20230816T0319Z.tar.gz __31__1891__20230820T0321Z.tar.gz __53__1105__20230723T0431Z.tar.gz
__1__11983__20230715T0116Z.tar.gz __31__639__20230813T0705Z.tar.gz __53__2025__20230820T1927Z.tar.gz
__1__12014__20230715T0839Z.tar.gz __33__1111__20230723T0150Z.tar.gz __53__906__20230720T1949Z.tar.gz
__1__12730__20230722T0925Z.tar.gz __33__1263__20230724T1657Z.tar.gz __57__1014__20230815T0707Z.tar.gz
__1__1625__20230818T1428Z.tar.gz __33__1557__20230818T0600Z.tar.gz __57__1311__20230816T2142Z.tar.gz
__13__1943__20230820T0952Z.tar.gz __33__242__20230811T0203Z.tar.gz __57__317__20230715T0121Z.tar.gz
__13__763__20230813T2304Z.tar.gz __33__995__20230815T0357Z.tar.gz __57__762__20230813T2315Z.tar.gz
__15__1867__20230819T2226Z.tar.gz __35__703__20230813T1444Z.tar.gz __5__9398__20230707T2025Z.tar.gz
__17__244__20230811T0232Z.tar.gz __35__713__20230813T1601Z.tar.gz __5__9582__20230708T0259Z/
__17__443__20230716T0333Z.tar.gz __41__1135__20230815T2218Z.tar.gz __5__9583__20230708T0301Z/
__17__480__20230716T1042Z.tar.gz __41__1565__20230818T0754Z.tar.gz __5__9584__20230708T0302Z/
__17__615__20230717T1808Z.tar.gz __41__914__20230814T1813Z.tar.gz __59__990__20230815T0358Z.tar.gz
__1__769__20230813T2313Z.tar.gz __43__1458__20230817T1614Z.tar.gz __61__1031__20230722T0821Z.tar.gz
__19__237__20230811T0200Z.tar.gz __43__1976__20230820T1304Z.tar.gz __61__108__20230712T2332Z.tar.gz
__1__9434__20230707T1411Z/ __45__122__20230712T2323Z.tar.gz __61__1881__20230820T0033Z.tar.gz
__1__9546__20230707T1919Z/ __45__1458__20230817T1618Z/ __61__359__20230811T1710Z.tar.gz
__19__659__20230813T1001Z.tar.gz __45__1458__20230817T1618Z.tar.gz __61__666__20230813T1013Z.tar.gz
__1__9840__20230708T2244Z.tar.gz __47__1141__20230815T2218Z.tar.gz __63__1903__20230820T0331Z.tar.gz
__1__9899__20230709T0025Z/ __47__1458__20230817T1613Z.tar.gz __63__2183__20230821T1552Z.tar.gz
__21__1245__20230724T1421Z.tar.gz __47__1848__20230819T2002Z.tar.gz __7__798__20230814T0324Z.tar.gz
__21__418__20230715T2322Z.tar.gz __47__242__20230811T0205Z.tar.gz __9__1654__20230818T1839Z.tar.gz
__23__565__20230812T2025Z.tar.gz __49__1244__20230724T1656Z.tar.gz __9__243__20230811T0212Z.tar.gz
__25__1568__20230818T0803Z.tar.gz __49__131__20230712T2337Z.tar.gz __9__445__20230715T2043Z.tar.gz
__25__1894__20230820T0320Z.tar.gz __49__1622__20230818T1425Z.tar.gz segment-stats.ser
__25__240__20230811T0207Z.tar.gz __5__12051__20230712T2046Z.tar.gz _tmp/
__29__157__20230713T1035Z.tar.gz __5__12076__20230713T0023Z.tar.gz tmp/
__29__1588__20230818T1133Z.tar.gz __51__700__20230813T1436Z.tar.gz
```
Contributor guide
Research direction
Start by tracing where the temporary .tar.gz files are created and retained in realtime table data directories. Determine the safe cleanup conditions and verify that periodic cleanup removes leftover archives without affecting active work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100