Azure / Azure/app-service-linux-docs
Stale output.tar.zst from prior deploy not cleaned up; runtime extracts orphan over current output.tar.gz
- Dominant language
- Java
- Stars
- 150
- Forks
- 90
- Avg merge
- 2m
- Merged PRs (30d)
- 1
Description
## Summary
On Linux Python App Service, when consecutive deploys produce different compression formats (`output.tar.zst` from one, `output.tar.gz` from a later one), both files persist in `/home/site/wwwroot/`. The Oryx runtime startup script extracts the OLDER `output.tar.zst` instead of the newer `output.tar.gz`, causing production to serve stale code with no error indication.
## Environment
- App Service: `wapd-image-finder` (Linux, Python 3.11)
- Image tag: `appsvc/python:3.11_20260203.2.tuxprod`
- Oryx Version: `0.2.20251017.2`
- Date observed: 2026-05-10 UTC
- Region: East US
## Reproduction sequence
1. Deploy A produces `output.tar.zst` (zstd-compressed) -- in our case a deploy attempt where `COMPRESS_DESTINATION_DIR=false` was set as an app setting but ignored by Oryx (separate issue, see microsoft/Oryx#2921).
2. Deploy B (later, via GitHub Actions OneDeploy) produces `output.tar.gz` (gzip).
3. Both files now sit in `/home/site/wwwroot/`. Microsoft does not delete the older one.
4. Container restarts. Runtime startup script extracts `output.tar.zst` (older) rather than `output.tar.gz` (current).
5. Production serves the OLDER deploy's source/static-assets.
## Evidence
```
$ ls -la /home/site/wwwroot/
-rwxrwxrwx 1 nobody nogroup 261 May 10 04:47 oryx-manifest.toml
-rwxrwxrwx 1 nobody nogroup 289966627 May 10 04:47 output.tar.gz <- current deploy
-rwxrwxrwx 1 nobody nogroup 277256844 May 10 02:23 output.tar.zst <- orphan from prior deploy
$ tar -tzf /home/site/wwwroot/output.tar.gz | grep assets/index
./frontend/dist/assets/index-w8ghkh8t.css
./frontend/dist/assets/index-DNf7ivpY.js <- new bundle hash, 404 in production
$ zstd -d -c /home/site/wwwroot/output.tar.zst | tar -t | grep assets/index
./frontend/dist/assets/index-BRgwFiec.js <- old bundle hash, 200 in production
./frontend/dist/assets/index-CjFrALqX.css
```
`oryx-manifest.toml` from the latest deploy says `CompressDestinationDir="true"`. The startup script appears to scan for any `output.tar.*` file rather than reading the manifest's intended format, picking the older `.zst` over the newer `.gz`.
## Workaround
Manually `rm /home/site/wwwroot/output.tar.zst` via Kudu shell, then `az webapp restart`. After the restart, runtime extracts only `output.tar.gz` and production serves the current deploy.
## Asks
1. Oryx runtime startup script should either (a) trust `oryx-manifest.toml`'s declared format and only look for the matching tarball, or (b) prefer the newest `output.tar.*` file by mtime.
2. Oryx build phase should clean up prior `output.tar.*` artifacts in `wwwroot` so only the newest exists after each deploy.
3. Fix `COMPRESS_DESTINATION_DIR=false` honoring (currently ignored by Oryx, tracked in microsoft/Oryx#2921). With that env var working, customers wouldn't end up with mixed-format tarballs in the first place.
## Related Microsoft issues we've filed today
- microsoft/Oryx#2921 -- Container panic when build writes `.zst` but runtime expects `.gz`
- Azure/app-service-linux-docs#262 -- OneDeploy 503 Service Unavailable post-auth on Linux Python deploys
This is the third distinct App Service Linux regression we've hit in a 24h window. Would appreciate triage.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.