Sporadic intermediate result leak in Circle CI
- Dominant language
- C
- Stars
- 12.8k
- Forks
- 794
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 31
Description
observed insert..select with re-partitioning leak a file in Circle CI:
```
--- /home/circleci/project/src/test/regress/expected/ensure_no_intermediate_data_leak.out.modified 2020-06-12 18:23:30.745763054 +0000
+++ /home/circleci/project/build-11/src/test/regress/results/ensure_no_intermediate_data_leak.out.modified 2020-06-12 18:23:30.749763152 +0000
@@ -19,21 +19,22 @@
SELECT * FROM run_command_on_workers($$
WITH xact_dirs AS (
SELECT pg_ls_dir('base/pgsql_job_cache') dir WHERE citus_version() NOT ILIKE '%windows%'
), result_files AS (
SELECT dir, pg_ls_dir('base/pgsql_job_cache/' || dir) result_file FROM xact_dirs
)
SELECT array_agg((xact_dirs.dir, result_files.result_file)) FROM xact_dirs LEFT OUTER JOIN result_files ON xact_dirs.dir = result_files.dir;
$$) WHERE result <> '';
nodename | nodeport | success | result
-----------+----------+---------+--------------------------------------------------------------------------
-(0 rows)
+ localhost | 57637 | t | {"(10_0_445,repartitioned_results_20149436507_from_13300003_to_1.data)"}
+(1 row)
```
Contributor guide
Research direction
Start with the regression test under src/test/regress/ensure_no_intermediate_data_leak and its expected output. Reproduce the insert..select with re-partitioning in Circle CI or the test environment, then trace the reported repartitioned_results file. Done means the test no longer observes an intermediate result file and the regression output is clean.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql, sql
- Domain
- databases, distributed-systems, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100