GoogleCloudPlatform / GoogleCloudPlatform/gcsfuse

Feature Request: Warn and block writes when a directory exceeds a configurable entry limit

Open
#4,696 6 comments 0 reactions 1 assignee Claimed by @PranjalC100 View on GitHub
feature request
Dominant language
Go
Stars
2.3k
Forks
510
Avg merge
2d 17h
Merged PRs (30d)
35

Description

## Summary

When a GCS bucket directory contains a very large number of files, Cloud Storage
FUSE currently has no mechanism to warn operators or degrade gracefully. In
production we observe:

- **~50,000 files in a single directory** → significant read/list performance
degradation
- **100,000+ files in a single directory** → very poor performance that can
lead to filesystem state corruption, causing extended downtime and potential
file loss

There is also no way to prevent new files from being written into an already
over-populated directory, so the problem compounds silently until it causes an
outage.

## Current Behavior

`gcsfuse` performs a full `ListObjects` (paginated) on every `readdir()` call
(when `kernel-list-cache-ttl-secs=0`) or on first access within a TTL window.
With 100k objects under one prefix this means:

- 10+ sequential paginated GCS API calls per listing
- Unbounded memory growth in the metadata/stat cache
- No warning in logs that a directory is approaching a problematic size
- No way to prevent further writes into an already over-populated directory
- No documented hard limit — operators discover the problem only after
degradation or corruption occurs

## Requested Features

### 1. Write guard — block new file creation above a threshold

When a directory's entry count (known from the stat/list cache, or from a
lightweight `ListObjects` prefix count) meets or exceeds a configurable limit,
reject `O_CREAT` operations on that directory with `ENOSPC` and emit a
structured `WARNING` log.

```yaml
file-system:
max-dir-entries: 50000 # block new creates and warn when parent dir hits this
```

Returning `ENOSPC` gives applications and scripts a clear, standard error
signal (`No space left on device`) rather than silently allowing the directory
to grow into the corruption zone.

### 2. Warning threshold log (pre-limit signal)

Emit a structured `WARNING` log when a `ListObjects` response for a single
directory prefix exceeds a lower configurable threshold, giving operators
time to act before the hard limit is reached.

```yaml
file-system:
large-dir-warning-threshold: 10000 # log WARNING when readdir exceeds this
```

### 3. Streaming / incremental readdir

Instead of buffering the entire `ListObjects` result before returning to the
kernel, stream entries back incrementally. This caps peak memory usage and
allows the kernel to begin iterating entries before the full GCS list
completes — directly addressing latency and OOM risk at extreme scale.

### 4. Documentation of limits

Publish a supported maximum directory size (entries per prefix) in the
official documentation with explicit guidance on mitigation strategies
(subdirectory sharding, HNS buckets, etc.).

## Why the write guard matters

Read-path mitigations (kernel list cache TTL, larger stat cache) reduce the
frequency of expensive listing but do not stop directories from growing further.
Without a write guard, every new file written into an already over-populated
directory makes the next `readdir` more expensive and moves the mount closer to
the corruption threshold. A write guard closes this feedback loop at the source.

## Impact

This affects any multi-tenant or user-generated-content workload where
individual users can accumulate files in a flat directory structure without
architectural guardrails. The lack of early warning means operators only
discover the limit under production load, at which point recovery requires
unmounting and remounting — causing service interruption.

## Workaround

Currently we rely on a combination of:
- `kernel-list-cache-ttl-secs: 30` to reduce frequency of full listings
- `stat-cache-max-size-mb: 128` to size the cache for 100k entries

These mitigate the performance impact but do not prevent corruption at
extreme file counts, and require operators to know the limits in advance.

## Environment

- Cloud Storage FUSE version: 2.x (config-file mode)
- Mount mode: `--config-file` with `implicit-dirs: true`
- GCS bucket type: standard (non-HNS)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.