microsoft / microsoft/Oryx

One of three Deploy Azure App Service tasks taking 3 times longer to deploy

Open
#2,590 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C#
Stars
881
Forks
195
Avg merge
20h 13m
Merged PRs (30d)
13

Description

One of our three Release Pipelines, using the task Deploy Azure App Service, is taking 3 times longer to deploy. We seem to have isolated it to Oryx.

Each Release Pipeline is deploying to its own Linux App Service within the same Subscription, using the same plan, using the same Agent, using the same Service Connection and using an artifact coming from the same Git hash (approx 108MB). The Release Pipelines are essentially also configured the same.

Example of SBX, taking 265 seconds:

[
    {
        "log_time": "2025-05-08T17:03:51.8178244Z",
        "id": "",
        "message": "Command: oryx build /tmp/zipdeploy/extracted -o /home/site/wwwroot --platform php --platform-version 8.2 -p virtualenv_name= --log-file /tmp/build-debug.log  -i /tmp/8dd8e524e831500 | tee /tmp/oryx-build.log",
        "type": 0,
        "details_url": null
    },
    {
        "log_time": "2025-05-08T17:03:52.5043238Z",
        "id": "",
        "message": "Operation performed by Microsoft Oryx, https://github.com/Microsoft/Oryx",
        "type": 0,
        "details_url": null
    },
    {
        "log_time": "2025-05-08T17:03:52.5237787Z",
        "id": "",
        "message": "You can report issues at https://github.com/Microsoft/Oryx/issues",
        "type": 0,
        "details_url": null
    },
    {
        "log_time": "2025-05-08T17:03:52.5376572Z",
        "id": "",
        "message": "",
        "type": 0,
        "details_url": null
    },
    {
        "log_time": "2025-05-08T17:03:52.5492187Z",
        "id": "",
        "message": "Oryx Version: 0.2.20250107.1+ef3fb2f9c490a4fbb1f76b5746cd4ba2ff0409f4, Commit: ef3fb2f9c490a4fbb1f76b5746cd4ba2ff0409f4, ReleaseTagName: 20250107.1",
        "type": 0,
        "details_url": null
    },
    {
        "log_time": "2025-05-08T17:03:52.5696109Z",
        "id": "",
        "message": "",
        "type": 0,
        "details_url": null
    },
    {
        "log_time": "2025-05-08T17:03:52.586482Z",
        "id": "",
        "message": "Build Operation ID: ff07ff251183d1fa",
        "type": 0,
        "details_url": null
    },
    {
        "log_time": "2025-05-08T17:03:52.5997201Z",
        "id": "",
        "message": "Repository Commit : 00bbbd5e-5ebc-4915-9536-d672eac3e94f",
        "type": 0,
        "details_url": null
    },
    {
        "log_time": "2025-05-08T17:03:52.6110734Z",
        "id": "",
        "message": "OS Type           : bullseye",
        "type": 0,
        "details_url": null
    },
    {
        "log_time": "2025-05-08T17:03:52.6229955Z",
        "id": "",
        "message": "Image Type        : githubactions",
        "type": 0,
        "details_url": null
    },
    {
        "log_time": "2025-05-08T17:03:52.6412193Z",
        "id": "",
        "message": "",
        "type": 0,
        "details_url": null
    },
    {
        "log_time": "2025-05-08T17:03:52.6557512Z",
        "id": "",
        "message": "Detecting platforms...",
        "type": 0,
        "details_url": null
    },
    {
        "log_time": "2025-05-08T17:03:56.7534737Z",
        "id": "",
        "message": "Detected following platforms:",
        "type": 0,
        "details_url": null
    },
    {
        "log_time": "2025-05-08T17:03:56.8297675Z",
        "id": "",
        "message": "  python: 3.8.20",
        "type": 0,
        "details_url": null
    },
    {
        "log_time": "2025-05-08T17:03:56.8517636Z",
        "id": "",
        "message": "  php: 8.2.28",
        "type": 0,
        "details_url": null
    },
    {
        "log_time": "2025-05-08T17:03:56.8871462Z",
        "id": "",
        "message": "",
        "type": 0,
        "details_url": null
    },
    {
        "log_time": "2025-05-08T17:03:56.9153882Z",
        "id": "",
        "message": "Using intermediate directory '/tmp/8dd8e524e831500'.",
        "type": 0,
        "details_url": null
    },
    {
        "log_time": "2025-05-08T17:03:56.9396224Z",
        "id": "",
        "message": "",
        "type": 0,
        "details_url": null
    },
    {
        "log_time": "2025-05-08T17:03:56.9615046Z",
        "id": "",
        "message": "Copying files to the intermediate directory...",
        "type": 0,
        "details_url": null
    },
    {
        "log_time": "2025-05-08T17:04:04.6875426Z",
        "id": "",
        "message": "Done in 8 sec(s).",
        "type": 0,
        "details_url": null
    },
    {
        "log_time": "2025-05-08T17:04:04.7243173Z",
        "id": "",
        "message": "",
        "type": 0,
        "details_url": null
    },
    {
        "log_time": "2025-05-08T17:04:04.7433202Z",
        "id": "",
        "message": "Source directory     : /tmp/8dd8e524e831500",
        "type": 0,
        "details_url": null
    },
    {
        "log_time": "2025-05-08T17:04:04.7616645Z",
        "id": "",
        "message": "Destination directory: /home/site/wwwroot",
        "type": 0,
        "details_url": null
    },
    {
        "log_time": "2025-05-08T17:04:04.7758728Z",
        "id": "",
        "message": "",
        "type": 0,
        "details_url": null
    },
    {
        "log_time": "2025-05-08T17:04:04.8447058Z",
        "id": "",
        "message": "PHP executable: /tmp/oryx/platforms/php/8.2.28/bin/php",
        "type": 0,
        "details_url": null
    },
    {
        "log_time": "2025-05-08T17:04:04.8617244Z",
        "id": "",
        "message": "No 'composer.json' file found; not running 'composer install'.",
        "type": 0,
        "details_url": null
    },
    {
        "log_time": "2025-05-08T17:04:04.9022501Z",
        "id": "",
        "message": "Preparing output...",
        "type": 0,
        "details_url": null
    },
    {
        "log_time": "2025-05-08T17:04:04.9185118Z",
        "id": "",
        "message": "",
        "type": 0,
        "details_url": null
    },
{
    "log_time": "2025-05-08T17:04:04.9408063Z",
    "id": "",
    "message": "Copying files to destination directory '/home/site/wwwroot'...",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-08T17:08:29.1502207Z",
    "id": "",
    "message": "Done in 265 sec(s).",
    "type": 0,
    "details_url": null
},
    {
        "log_time": "2025-05-08T17:08:29.1930507Z",
        "id": "",
        "message": "",
        "type": 0,
        "details_url": null
    },
    {
        "log_time": "2025-05-08T17:08:29.2152905Z",
        "id": "",
        "message": "Removing existing manifest file",
        "type": 0,
        "details_url": null
    },
    {
        "log_time": "2025-05-08T17:08:29.2457195Z",
        "id": "",
        "message": "Creating a manifest file...",
        "type": 0,
        "details_url": null
    },
    {
        "log_time": "2025-05-08T17:08:29.2705275Z",
        "id": "",
        "message": "Manifest file created.",
        "type": 0,
        "details_url": null
    },
    {
        "log_time": "2025-05-08T17:08:29.2905518Z",
        "id": "",
        "message": "Copying .ostype to manifest output directory.",
        "type": 0,
        "details_url": null
    },
    {
        "log_time": "2025-05-08T17:08:29.3099985Z",
        "id": "",
        "message": "",
        "type": 0,
        "details_url": null
    },
    {
        "log_time": "2025-05-08T17:08:29.3315968Z",
        "id": "",
        "message": "Done in 273 sec(s).",
        "type": 0,
        "details_url": null
    }
]

Example of DEV taking 3-4x the amount of time, 868 seconds:

[
{
    "log_time": "2025-05-09T14:09:56.1805533Z",
    "id": "",
    "message": "Command: oryx build /tmp/zipdeploy/extracted -o /home/site/wwwroot --platform php --platform-version 8.2 -p virtualenv_name= --log-file /tmp/build-debug.log  -i /tmp/8dd8f032cd47564 | tee /tmp/oryx-build.log",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:09:56.7937999Z",
    "id": "",
    "message": "Operation performed by Microsoft Oryx, https://github.com/Microsoft/Oryx",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:09:56.819967Z",
    "id": "",
    "message": "You can report issues at https://github.com/Microsoft/Oryx/issues",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:09:56.8310938Z",
    "id": "",
    "message": "",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:09:56.8430599Z",
    "id": "",
    "message": "Oryx Version: 0.2.20250107.1+ef3fb2f9c490a4fbb1f76b5746cd4ba2ff0409f4, Commit: ef3fb2f9c490a4fbb1f76b5746cd4ba2ff0409f4, ReleaseTagName: 20250107.1",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:09:56.8593386Z",
    "id": "",
    "message": "",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:09:56.8744096Z",
    "id": "",
    "message": "Build Operation ID: e97017e546914a56",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:09:56.8859629Z",
    "id": "",
    "message": "Repository Commit : c7a1a230-8fdc-4e8e-9247-0d768766edaa",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:09:56.9006698Z",
    "id": "",
    "message": "OS Type           : bullseye",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:09:56.9196102Z",
    "id": "",
    "message": "Image Type        : githubactions",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:09:56.9344093Z",
    "id": "",
    "message": "",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:09:56.9516841Z",
    "id": "",
    "message": "Detecting platforms...",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:10:00.5774934Z",
    "id": "",
    "message": "Detected following platforms:",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:10:00.6185069Z",
    "id": "",
    "message": "  python: 3.8.20",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:10:00.6359339Z",
    "id": "",
    "message": "  php: 8.2.28",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:10:00.7765572Z",
    "id": "",
    "message": "",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:10:00.8303555Z",
    "id": "",
    "message": "Using intermediate directory '/tmp/8dd8f032cd47564'.",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:10:00.8568839Z",
    "id": "",
    "message": "",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:10:00.8816397Z",
    "id": "",
    "message": "Copying files to the intermediate directory...",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:10:09.7951363Z",
    "id": "",
    "message": "Done in 9 sec(s).",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:10:09.8289995Z",
    "id": "",
    "message": "",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:10:09.8491152Z",
    "id": "",
    "message": "Source directory     : /tmp/8dd8f032cd47564",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:10:09.8699031Z",
    "id": "",
    "message": "Destination directory: /home/site/wwwroot",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:10:09.8885427Z",
    "id": "",
    "message": "",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:10:10.0901955Z",
    "id": "",
    "message": "PHP executable: /tmp/oryx/platforms/php/8.2.28/bin/php",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:10:10.1136574Z",
    "id": "",
    "message": "No 'composer.json' file found; not running 'composer install'.",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:10:10.1360334Z",
    "id": "",
    "message": "Preparing output...",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:10:10.1574027Z",
    "id": "",
    "message": "",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:10:10.2159263Z",
    "id": "",
    "message": "Copying files to destination directory '/home/site/wwwroot'...",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:24:38.0307155Z",
    "id": "",
    "message": "Done in 868 sec(s).",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:24:38.0809035Z",
    "id": "",
    "message": "",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:24:38.0935092Z",
    "id": "",
    "message": "Removing existing manifest file",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:24:38.1091884Z",
    "id": "",
    "message": "Creating a manifest file...",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:24:38.1228719Z",
    "id": "",
    "message": "Manifest file created.",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:24:38.13774Z",
    "id": "",
    "message": "Copying .ostype to manifest output directory.",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:24:38.1551968Z",
    "id": "",
    "message": "",
    "type": 0,
    "details_url": null
},
{
    "log_time": "2025-05-09T14:24:38.170181Z",
    "id": "",
    "message": "Done in 878 sec(s).",
    "type": 0,
    "details_url": null
}
]

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by comparing the supplied Oryx build logs for the SBX and DEV Azure App Service deployments, especially the time spent copying files to the destination directory. Investigate why equivalent pipelines using the same artifact and plan have different durations; done means identifying the cause and documenting a reproducible fix or next diagnostic step.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, php, python
Domain
build-system, cloud, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.