aws / aws/chalice

FileNotFound not properly handled in filewatcher

Open
#997 2 comments 2 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
11.1k
Forks
1k
Avg merge
1d 22h
Merged PRs (30d)
2

Description

The filewatcher will get an exception in a situation where a symlink to non-existent file exists in the directory it is watching. This will occur because `mtime` method in `utils.py` uses `os.stat` which will fail with `FileNotFound` exception if a symlink that does not point to a real file is detected.

This is pretty easily occurring situation, for example Emacs creates an `.#app.py` file symlinked to a non-existing file in the directory when editing. This is what then happens:

```
$ chalice local
Exception in thread Thread-2:
Traceback (most recent call last):
File "/Users/santtu/.pyenv/versions/3.6.6/lib/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/Users/santtu/.pyenv/versions/3.6.6/lib/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/Users/santtu/.pyenv/versions/3.6.6/envs/adc-reborn/lib/python3.6/site-packages/chalice/cli/filewatch/stat.py", line 49, in poll_for_changes_until_shutdown
self._seed_mtime_cache(root_dir)
File "/Users/santtu/.pyenv/versions/3.6.6/envs/adc-reborn/lib/python3.6/site-packages/chalice/cli/filewatch/stat.py", line 59, in _seed_mtime_cache
self._mtime_cache[path] = self._osutils.mtime(path)
File "/Users/santtu/.pyenv/versions/3.6.6/envs/adc-reborn/lib/python3.6/site-packages/chalice/utils.py", line 249, in mtime
return os.stat(path).st_mtime
FileNotFoundError: [Errno 2] No such file or directory: '/Users/santtu/dev/adc-reborn/adc-lambda/.#app.py'

Serving on 127.0.0.1:8000
```

The file watcher (reloader) will not work from this point on, e.g. any changes to application files won't be reloaded.

You don't need Emacs for this, just run `ln -s foo bar` in the app directory before or during `chalice local`.

See this change which will cause `test_stat.py` test to fail: https://github.com/santtu/chalice/commit/ade2a5d6ed9b7af30980caabfa09c1ed5b66a635

(I do not understand the rationale on the `initial_scan` in `FakeOSUtils` --- it seems to assume something on the inner workings of `StatFileWatcher`.)

Contributor guide

Open the contributing guide

Research direction

Start with mtime in utils.py and the file-watcher tests in test_stat.py, especially FakeOSUtils and StatFileWatcher. Reproduce the broken-symlink case described in the issue, then run the relevant tests. Done means the watcher handles a missing symlink target without stopping and the test suite passes.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devtools
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.