clock skew in `wsl2`
- Dominant language
- C++
- Stars
- 33.7k
- Forks
- 1.8k
- Avg merge
- 3d 17h
- Merged PRs (30d)
- 116
Description
### Windows Version
Microsoft Windows [Version 10.0.22631.3737]
### WSL Version
2.2.4.0
### Are you using WSL 1 or WSL 2?
- [X] WSL 2
- [ ] WSL 1
### Kernel Version
5.15.153.1-2
### Distro Version
CBL-Mariner 2.0.20240628
### Other Software
_No response_
### Repro Steps
1. Start the Linux instance.
2. Do various things in the `wsl` instance.
3. Observe that, occasionally, you get error messages regarding timestamps in the future, and detection of clock skew. Sometimes, depending on the task, clock skew occurs often enough that you can't complete the task (e.g. building RPMs wherein the tests rely on file modification times that end up appearing to be in the future).
### Expected Behavior
I expect to observe clock skew quite rarely, and to have it impact my work on a much less frequent basis.
### Actual Behavior
Here are a couple examples.
`dmesg` output
```
[danderson: /tmp/tmpawdfb76t]
> dmesg | tail -n 40
[404082.973889] systemd-journald[84]: Time jumped backwards, rotating.
[404111.348115] systemd-journald[84]: Time jumped backwards, rotating.
[404139.703269] systemd-journald[84]: Time jumped backwards, rotating.
[404168.096403] systemd-journald[84]: Time jumped backwards, rotating.
[404196.480773] systemd-journald[84]: Time jumped backwards, rotating.
[404224.872429] systemd-journald[84]: Time jumped backwards, rotating.
[404253.223833] systemd-journald[84]: Time jumped backwards, rotating.
[404281.594155] systemd-journald[84]: Time jumped backwards, rotating.
[404309.968480] systemd-journald[84]: Time jumped backwards, rotating.
[404338.363311] systemd-journald[84]: Time jumped backwards, rotating.
[404366.732147] systemd-journald[84]: Time jumped backwards, rotating.
[404395.108002] systemd-journald[84]: Time jumped backwards, rotating.
[404423.480880] systemd-journald[84]: Time jumped backwards, rotating.
[404451.831353] systemd-journald[84]: Time jumped backwards, rotating.
[404480.194392] systemd-journald[84]: Time jumped backwards, rotating.
[404508.549149] systemd-journald[84]: Time jumped backwards, rotating.
[404536.920285] systemd-journald[84]: Time jumped backwards, rotating.
[404565.283479] systemd-journald[84]: Time jumped backwards, rotating.
[404593.654135] systemd-journald[84]: Time jumped backwards, rotating.
[404622.024996] systemd-journald[84]: Time jumped backwards, rotating.
[404650.397592] systemd-journald[84]: Time jumped backwards, rotating.
[404678.775581] systemd-journald[84]: Time jumped backwards, rotating.
[404707.127293] systemd-journald[84]: Time jumped backwards, rotating.
[404735.490979] systemd-journald[84]: Time jumped backwards, rotating.
[404763.840208] systemd-journald[84]: Time jumped backwards, rotating.
[404792.189098] systemd-journald[84]: Time jumped backwards, rotating.
[404820.536595] systemd-journald[84]: Time jumped backwards, rotating.
[404848.904155] systemd-journald[84]: Time jumped backwards, rotating.
[404877.268762] systemd-journald[84]: Time jumped backwards, rotating.
[404905.630039] systemd-journald[84]: Time jumped backwards, rotating.
[404933.999577] systemd-journald[84]: Time jumped backwards, rotating.
[404962.374937] systemd-journald[84]: Time jumped backwards, rotating.
[404990.729142] systemd-journald[84]: Time jumped backwards, rotating.
[405019.091547] systemd-journald[84]: Time jumped backwards, rotating.
[405047.445148] systemd-journald[84]: Time jumped backwards, rotating.
[405075.793623] systemd-journald[84]: Time jumped backwards, rotating.
[405104.138690] systemd-journald[84]: Time jumped backwards, rotating.
[405132.488783] systemd-journald[84]: Time jumped backwards, rotating.
[405160.846706] systemd-journald[84]: Time jumped backwards, rotating.
[405189.185044] systemd-journald[84]: Time jumped backwards, rotating.
```
Partial (relevant) output when `meson` tests fail, which are run as part of building binary RPMs:
```
Traceback (most recent call last):
File "/builddir/build/BUILD/meson-1.3.1/meson.py", line 40, in
sys.exit(mesonmain.main())
File "/builddir/build/BUILD/meson-1.3.1/mesonbuild/mesonmain.py", line 294, in main
return run(sys.argv[1:], launcher)
File "/builddir/build/BUILD/meson-1.3.1/mesonbuild/mesonmain.py", line 280, in run
return errorhandler(e, 'setup')
File "/builddir/build/BUILD/meson-1.3.1/mesonbuild/mesonmain.py", line 40, in errorhandler
raise e
File "/builddir/build/BUILD/meson-1.3.1/mesonbuild/mesonmain.py", line 278, in run
return msetup.run(['--reconfigure'] + args[2:])
File "/builddir/build/BUILD/meson-1.3.1/mesonbuild/msetup.py", line 358, in run
app.generate()
File "/builddir/build/BUILD/meson-1.3.1/mesonbuild/msetup.py", line 181, in generate
return self._generate(env, capture, vslite_ctx)
File "/builddir/build/BUILD/meson-1.3.1/mesonbuild/msetup.py", line 251, in _generate
captured_compile_args = intr.backend.generate(capture, vslite_ctx)
File "/builddir/build/BUILD/meson-1.3.1/mesonbuild/backend/ninjabackend.py", line 657, in generate
self.generate_ending()
File "/builddir/build/BUILD/meson-1.3.1/mesonbuild/backend/ninjabackend.py", line 3679, in generate_ending
deps = self.get_regen_filelist()
File "/builddir/build/BUILD/meson-1.3.1/mesonbuild/backend/backends.py", line 1353, in get_regen_filelist
self.check_clock_skew(deps)
File "/builddir/build/BUILD/meson-1.3.1/mesonbuild/backend/backends.py", line 1380, in check_clock_skew
raise MesonException(f'Clock skew detected. File {absf} has a time stamp {delta:.4f}s in the future.')
mesonbuild.utils.core.MesonException: Clock skew detected. File /builddir/build/BUILD/meson-1.3.1/b 41766a13e6/../test cases/common/22 object extraction/meson.build has a time stamp 0.2633s in the future.
ninja: error: rebuilding 'build.ninja': subcommand failed
```
(In the build log this excerpt is from, there were 32 such clock skew failures. I've tried building this package five times today, and each instance results in a similar amount of failures.)
Note that I've seen the issues about clock skew after sleeping. A few things make me think this is different:
1. I'm running a version of WSL that should have the fix for that issue.
2. I observe these issues right after restarting my computer, prior to the computer sleeping.
3. Running `sudo hwclock -s` doesn't seem to change anything.
### Diagnostic Logs
_No response_
Contributor guide
Research direction
Start by reproducing the issue in the reported WSL 2 environment after restarting Windows, using dmesg and a Meson build to observe backward jumps and future file timestamps. Compare the reported behavior with WSL's clock-synchronization path and the documented post-sleep fix. Done means repeated builds no longer encounter clock-skew failures under the described conditions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- linux
- Domain
- build-system, operating-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100