TraceMachina / TraceMachina/nativelink

Potential sanitizer issue in filesystem_store_test

Open
#1,102 0 comments 0 reactions 1 assignee View on GitHub

@MarcusSorealheis is already working on this.

Since Aug 12, 2024.

bug
Dominant language
Rust
Stars
1.6k
Forks
244
Avg merge
1d 16h
Merged PRs (30d)
54

Description

This occurred during an unrelated CI run (https://github.com/TraceMachina/nativelink/actions/runs/9831704252/job/27139463553?pr=1100):

FAIL: //nativelink-store:integration_tests/filesystem_store_test (see /home/runner/.bazel/execroot/_main/bazel-out/k8-fastbuild/testlogs/nativelink-store/integration_tests/filesystem_store_test/test.log)
[1,545 / 1,570] 25 / 47 tests, 1 failed; Testing //nativelink-store:integration_tests/shard_store_test; 6s linux-sandbox ... (4 actions running)
INFO: From Testing //nativelink-store:integration_tests/filesystem_store_test:
==================== Test output for //nativelink-store:integration_tests/filesystem_store_test:

running 19 tests
test eviction_drops_file_test ... ok
test digest_contents_replaced_continues_using_old_data ... ok
  2024-07-08T00:59:12.972929Z ERROR nativelink_store::filesystem_store: Failed to touch file, err: Error { code: NotFound, messages: ["No such file or directory (os error 2)", "Failed to touch file in filesystem store \"/tmp/bazel-working-directory/_main/_tmp/4570b71ef2441e1548bbbb5c539c5a14/8686312440345158165/content_path/0123456789abcdef000000000000000000010000000000000123456789abcdef-10\""] }
    at nativelink-store/src/filesystem_store.rs:339

  2024-07-08T00:59:12.973502Z  WARN nativelink_store::filesystem_store: Failed to rename file, digest: DigestInfo { size_bytes: 10, hash: "0123456789abcdef000000000000000000010000000000000123456789abcdef" }, from_path: "/tmp/bazel-working-directory/_main/_tmp/4570b71ef2441e1548bbbb5c539c5a14/8686312440345158165/content_path/0123456789abcdef000000000000000000010000000000000123456789abcdef-10", to_path: "/tmp/bazel-working-directory/_main/_tmp/4570b71ef2441e1548bbbb5c539c5a14/1726301081[529](https://github.com/TraceMachina/nativelink/actions/runs/9831704252/job/27139463553?pr=1100#step:4:530)326313/temp_path/0123456789abcdef000000000000000000010000000000000500000000000000-10", err: Error { code: NotFound, messages: ["No such file or directory (os error 2)"] }
    at nativelink-store/src/filesystem_store.rs:366

test deleted_file_removed_from_store ... ok
test atime_updates_on_get_part_test ... ok
test eviction_on_insert_calls_unref_once ... ok
test file_continues_to_stream_on_content_replace_test ... ok
test file_gets_cleans_up_on_cache_eviction ... FAILED
test get_part_timeout_test ... ok
test get_file_size_uses_block_size ... ok
test get_part_is_zero_digest ... ok
test has_with_results_on_zero_digests ... ok
  2024-07-08T00:59:15.319226Z  WARN nativelink_store::filesystem_store: Failed to delete file, path: "/tmp/bazel-working-directory/_main/_tmp/4570b71ef2441e1548bbbb5c539c5a14/1683231632[537](https://github.com/TraceMachina/nativelink/actions/runs/9831704252/job/27139463553?pr=1100#step:4:538)1328731/temp_path/0123456789abcdef000000000000000000020000000000001500000000000000-10", err: Error { code: NotFound, messages: ["No such file or directory (os error 2)"] }
    at nativelink-store/src/filesystem_store.rs:511

test oldest_entry_evicted_with_access_times_loaded_from_disk ... ok
  2024-07-08T00:59:15.352572Z ERROR nativelink_store::filesystem_store: Failed to rename file, err: Error { code: NotFound, messages: ["No such file or directory (os error 2)", "Failed to rename temp file to final path \"/tmp/bazel-working-directory/_main/_tmp/4570b71ef2441e1548bbbb5c539c5a14/11890447677042904716/content_path/0123456789abcdef000000000000000000010000000000000123456789abcdef-10\""] }, from_path: "/tmp/bazel-working-directory/_main/_tmp/4570b71ef2441e1548bbbb5c[539](https://github.com/TraceMachina/nativelink/actions/runs/9831704252/job/27139463553?pr=1100#step:4:540)c5a14/7757332695299337649/temp_path/0123456789abcdef000000000000000000010000000000001600000000000000-10", final_path: "/tmp/bazel-working-directory/_main/_tmp/4570b71ef2441e1548bbbb5c539c5a14/11890447677042904716/content_path/0123456789abcdef000000000000000000010000000000000123456789abcdef-10"
    at nativelink-store/src/filesystem_store.rs:694
    in nativelink_store::filesystem_store::filesystem_store_emplace_file

test rename_on_insert_fails_due_to_filesystem_error_proper_cleanup_happens ... ok
test temp_files_get_deleted_on_replace_test ... ok
test update_file_future_drops_before_rename ... ok
test update_with_whole_file_closes_file ... ok
test update_with_whole_file_slow_path_when_low_file_descriptors ... ok
test update_with_whole_file_uses_same_inode ... ok
test valid_results_after_shutdown_test ... ok

failures:

---- file_gets_cleans_up_on_cache_eviction stdout ----
Error: Error { code: Internal, messages: ["Sender dropped before sending EOF", "During first read of buf_channel::take()", "Error reading remaining bytes"] }


failures:
    file_gets_cleans_up_on_cache_eviction

test result: FAILED. 18 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 2.85s

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.