awslabs / awslabs/filemoverexpress

Hot folder: guard the shared hotFolders map with a mutex

Open
#98 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
4
Forks
1
Avg merge
9h 54m
Merged PRs (30d)
41

Description

## Background

Follow-up from review of #96 (per-hot-folder debounce).

The package-level `hotFolders` map in `src/cli/core/upload/hot_folder/hot_folder.go` is read under `mtx` by `recordFileEvent` and `processPendingUploadsLocked`, but it is **mutated without holding `mtx`** by:

- `configureHotFolders`
- `ConfigureHotFolderWatcher`
- `RemoveOldHotFolders`
- `removeDuplicateNamedHotFolders`

This is a pre-existing latent data race (config edits touching the map concurrently with the watcher goroutine iterating it). #96 does not introduce it, but by iterating `hotFolders` under `mtx` in the debounce path it deepens the entanglement.

## Why not fixed in #96

A correct fix needs deliberate lock structuring, not a blanket `mtx.Lock()` in each writer: `ConfigureHotFolderWatcher` calls `removeDuplicateNamedHotFolders`, so naively locking both would self-deadlock (`mtx` is a non-reentrant `sync.Mutex`). Asbjorn flagged it as a non-blocking follow-up rather than expanding the debounce PR's scope.

## Proposed fix

Introduce a dedicated `sync.RWMutex` for `hotFolders` (separate from the `mtx` guarding `pendingUploads`/`lastUpdated`), take a read lock in the watcher/debounce readers and a write lock in the four writers above, and restructure the `ConfigureHotFolderWatcher` -> `removeDuplicateNamedHotFolders` call path so the lock is acquired exactly once (e.g. an unexported `*Locked` variant, matching the existing `processPendingUploadsLocked` convention). Run with `go test -race` to confirm.

Contributor guide

Open the contributing guide

Research direction

Start in src/cli/core/upload/hot_folder/hot_folder.go by tracing hotFolders access in recordFileEvent, processPendingUploadsLocked, configureHotFolders, ConfigureHotFolderWatcher, RemoveOldHotFolders, and removeDuplicateNamedHotFolders. Run go test -race while reviewing the lock paths; done means all map reads and writes are protected without self-deadlock, including the ConfigureHotFolderWatcher call chain.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
cli
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
75/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.