apache / apache/datafusion-comet

Object store cache key drops the ABFS container, so two containers in one storage account share a store instance

Open
#4,993 1 comment 0 reactions 1 assignee Claimed by @peterxcli View on GitHub
area:scan bug correctness help wanted priority:critical
Dominant language
Scala
Stars
1.3k
Forks
373
Avg merge
2d 4h
Merged PRs (30d)
198

Description

## Describe the bug

`prepare_object_store_with_configs` in `native/core/src/parquet/parquet_support.rs` caches object store instances under `(url_key, config_hash)`, where `url_key` is built from:

```rust
let url_key = format!(
"{}://{}",
scheme,
&url[url::Position::BeforeHost..url::Position::AfterPort],
);
```

`Position::BeforeHost` starts *after* the URL userinfo. For ABFS, the container is the userinfo, not the host:

```
abfss://container@account.dfs.core.windows.net/path/to/file.parquet
```

So `abfss://c1@acct.dfs.core.windows.net/...` and `abfss://c2@acct.dfs.core.windows.net/...` both produce the key `abfss://acct.dfs.core.windows.net`. `config_hash` is computed over the per-session Hadoop config map, which does not vary by container, so it does not disambiguate them either.

Meanwhile `MicrosoftAzureBuilder::parse_url` bakes the container into the store instance from the URL userinfo, and the `Path` handed to that store is container-relative (the bucket/container is stripped by `Path::from_url_path(url.path())` in `PhysicalPlanner::get_partitioned_files`).

The consequence: within one executor process, reading `abfss://c2@acct/data/f.parquet` after `abfss://c1@acct/data/f.parquet` should hit the cached `c1` store and read **c1's data**.

## Steps to reproduce

Not yet reproduced. This was found by code reading during review of an unrelated change. Confirming it needs an Azure storage account with two containers holding same-named paths, read from a single executor in one application.

## Expected behavior

Each ABFS container resolves to its own object store instance, and a read from one container never returns another container's data.

## Additional context

- S3, GCS, and HDFS are unaffected: their bucket/host lives in the URL host component, which `url_key` does keep.
- The same key derivation is now also used to key a shared Parquet metadata cache, where the impact is narrower (entries are additionally validated against file size and modification time). That is documented as a known limitation there; this issue is about the store cache itself, where no such validation exists.
- The object store cache is process-wide and unbounded by design, so a wrong entry persists for the executor's lifetime.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.