Azure / Azure/azure-functions-python-worker

Nondeterministc deploy bug with 5+ different non-descript failures on two different OS's, worked before, support stumped

Đang mở
#1,306 12 bình luận 0 reaction 1 người được giao Được giao cho @vrdmr Xem trên GitHub
area:python-functions needs-attention :wave:
Ngôn ngữ chính
Python
Star
357
Fork
116
Merge trung bình
32 phút
Pull request đã merge (30 ngày)
1

Mô tả

Unsure how to title this. It's been a week of debugging and isolating with no results.
Is this issue in the correct repo? Also unsure. Help me out here.
I'll walk you through my journey.

I have a very large code base, it runs flawlessly locally. I've deployed it just fine for a long time until recently.
I made a change, including `google-cloud-texttospeech` in `requirements.txt`.
It stopped deploying after this (maybe relevant, maybe not).
Removing this change, to the **exact same code base** as before, still fails to deploy.

Some errors I get at random, and I've tried incredibly hard to isolate.
Both development environments can deploy other things successfully, and push to a new app fine as well.
I have not made any local environment changes at all when it began to fail.

**Details:**
Azure Functions runtime version 4.24.4.4, Linux premium plan with Elastic Premium EP3.
I deploy directly from visual studio code azure function extension.
My python code follows the folder structure of the v1 coding model (no decorators, lots of folders), however I am on v2 (host.json etc.), this has always worked, and runs locally of course.
Deploying to python 3.9.7.

**First development environment:**
Manjaro linux, vscode 1.81.1, azure extension 1.12.3

Errors I've encountered seemingly at random, when trying to deploy:

```
2:50:41 PM debugApp123: **Deployment successful**. deployer = ms-azuretools-vscode deploymentPath = Functions App ZipDeploy. Extract zip. Remote build.
2:51:07 PM debugApp123: Syncing triggers...
2:51:12 PM debugApp123: Querying triggers...
2:51:18 PM debugApp123: **No HTTP triggers found.**
```

```
3:34:11 PM debugApp123: Deployment Failed. deployer = ms-azuretools-vscode deploymentPath = Functions App ZipDeploy. Extract zip. Remote build.
3:34:25 PM debugApp123: Deployment failed.
```

```
4:00:49 PM: Error: The operation was aborted.
```

```
10:52:10 AM ae-API-compute: Deployment successful. deployer = ms-azuretools-vscode deploymentPath = Functions App ZipDeploy. Extract zip. Remote build.
10:52:42 AM ae-API-compute: Syncing triggers...
10:52:49 AM ae-API-compute: Querying triggers...
10:52:52 AM ae-API-compute: WARNING: Some http trigger urls cannot be displayed in the output window because they require an authentication token. Instead, you may copy them from the Azure Functions explorer.
```
Except it was not successful and functions are empty / don't run (the zip file in webjobstorage contains them thought).

```
11:55:04 AM debugApp123: Syncing triggers...
11:55:45 AM debugApp123: Syncing triggers (Attempt 2/6)...
11:55:56 AM debugApp123: Syncing triggers (Attempt 3/6)...
11:56:17 AM debugApp123: Syncing triggers (Attempt 4/6)...
11:56:59 AM debugApp123: Syncing triggers (Attempt 5/6)...
11:58:20 AM debugApp123: Syncing triggers (Attempt 6/6)...
11:59:12 AM: Error: Encountered an error (ServiceUnavailable) from host runtime.
```

And a very exciting shiny rare one:
```
Offset to Central Directory cannot be held in an Int64.
```

**Second development environment:**
Windows, visual studio code, azure extension v1.12.4.
I've also used this environment before, and it has worked, same as the above one.

Errors:

One successful deploy, and randomly most of the above (with no changes), as well as a new one:

```
10:35:34 AM ae-API-compute: Starting deployment...
10:35:34 AM ae-API-compute: Creating zip package...
10:45:11 AM: Error: socket hang up
```

**Some other random things I've tried:**

- different development environments with different OS's, cloning the repo fresh
- rolling back azure function extension in visual studio code to various versions
- protobuf==3.20.*
- setting AzureWebJobsFeatureFlags:EnableWorkerIndexing
- setting SCM_DO_BUILD_DURING_DEPLOYMENT:1
- removing type hints
- restarting

**What now?**
The errors are not helpful. Logs are missing, I can drill down into events, insights, many different logs, they are all over the place and they are all useless or empty. I've mentioned that I've had multiple 2hr+ calls with the technical support team of various ever-increasing escalations, and they are just as stumped as me (and I am grateful for their efforts).

**Any thoughts?**

What do I try next? Any information I can provide?

**See also** (maybe relevant, I don't know at this point):
https://github.com/microsoft/vscode-azurefunctions/issues/3805
https://github.com/microsoft/vscode-azurefunctions/issues/2529
https://github.com/microsoft/Oryx/issues/1774
https://stackoverflow.com/questions/76478668/adding-python-module-google-cloud-storage-is-causing-a-working-azure-function-ap
https://stackoverflow.com/questions/72441758/typeerror-descriptors-cannot-not-be-created-directly
https://github.com/projectkudu/kudu/issues/3348
https://github.com/microsoft/azure-pipelines-tasks/issues/14201
https://github.com/microsoft/vscode-azurefunctions/issues/2529
https://github.com/microsoft/Oryx/issues/1774

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.