PhysicalFileProvider does not reload on change when a folder with a file does not exist and then moved to expected parent folder

# Issue Title

PhysicalFileProvider does not reload on change when a folder with a file does not exist and then moved to expected parent folder

# General

**Environment**
Windows 10 Pro 1903 18362.900
.NET Core 3.1.102
Microsoft Visual Studio Professional 2019 16.3.6

**Example**
[Test.zip](https://github.com/dotnet/core/files/4849660/Test.zip)

**Reproduction path**
1. Open attached solution. Pay attention at Program.cs where configuration file is added - `config.AddJsonFile(@"C:\Configuration\Folder\file.json", optional: true, reloadOnChange: true);`.
2. Create folder "Configuration" on disk C (It may be any other disk. If you are to use another disk, update code mentioned in first step)
3. Build and run Test project from the solution. It should log empty lines in the console, because there is no _file.json_ in _C:\Configuration\Folder_ yet.
4. **Move** (not Copy) folder "Folder" (in contains file.json inside) from the root of attached solution to the _C:\Configuration_

**Expected result**
Test application starts logging in the console "MyValue"

**Actual result**
Test application continues logging empty lines.

**Note**
When folder is **copied** to _C:\Configuration_, the application works as expected - it starts logging value from the file.json. So it seems like a PhysicalFileProvider is not able to detect when a folder with expected file is moved to parent folder.