Azure / Azure/azure-functions-host

Self-hosted v3 function app fails to run in K8S pod if filesystem is readonly

Open
#8,181 9 comments 0 reactions 1 assignee Claimed by @jainharsh98 View on GitHub
Dominant language
C#
Stars
2k
Forks
482
Avg merge
2d 12h
Merged PRs (30d)
38

Description

I’m hosting the Functions runtime in a dotnet Core 3.1 app to run as a Docker image inside a Kubernetes cluster.
However, the function app fails early on startup with the following stack trace:

```
System.IO.IOException: Read-only file system
at System.IO.FileSystem.CreateDirectory(String fullPath)
at System.IO.Directory.CreateDirectory(String path)
at System.IO.Abstractions.DirectoryWrapper.CreateDirectory(String path)
at Microsoft.Azure.WebJobs.Script.FileUtility.EnsureDirectoryExists(String path) in /src/azure-functions-host/src/WebJobs.Script/Extensions/FileUtility.cs:line 40
at Microsoft.Azure.WebJobs.Script.WebHost.FileMonitoringService.InitializeSecondaryFileWatchers() in /src/azure-functions-host/src/WebJobs.Script.WebHost/FileMonitoringService.cs:line 192
at Microsoft.Azure.WebJobs.Script.Utility.ExecuteAfterColdStartDelay(IEnvironment environment, Action targetAction, CancellationToken cancellationToken) in /src/azure-functions-host/src/WebJobs.Script/Utility.cs:line 728
at Microsoft.Azure.WebJobs.Script.WebHost.FileMonitoringService.InitializeFileWatchers() in /src/azure-functions-host/src/WebJobs.Script.WebHost/FileMonitoringService.cs:line 173
at Microsoft.Azure.WebJobs.Script.WebHost.FileMonitoringService.StartAsync(CancellationToken cancellationToken) in /src/azure-functions-host/src/WebJobs.Script.WebHost/FileMonitoringService.cs:line 94
at Microsoft.Extensions.Hosting.Internal.Host.StartAsync(CancellationToken cancellationToken)
at Microsoft.Azure.WebJobs.Script.WebHost.WebJobsScriptHostService.UnsynchronizedStartHostAsync(ScriptHostStartupOperation activeOperation, Int32 attemptCount, JobHostStartupMode startupMode) in /src/azure-functions-host/src/WebJobs.Script.WebHost/WebJobsScriptHostService.cs:line 292
```

#### Investigative information

The deployment manifest configures the pod with the following security context:

```yaml
securityContext:
allowPrivilegeEscalation: false
readOnlyRootFilesystem: true
runAsNonRoot: true
runAsUser: 10001
runAsGroup: 10001
```

The function app **does not host** any functions at all.
I tried narrowing down the root cause and it lead me to try with various triggers until I had to remove all functions from the app.
However, the error still occurs.

#### Repro steps

Here is the `Dockerfile` that builds the function when creating the image:

```Dockerfile
FROM mcr.microsoft.com/dotnet/core/sdk:3.1 AS installer-env

COPY ./FunctionApp /src/dotnet-function-app
COPY ./nuget.conf /src/dotnet-function-app/nuget.config
COPY ./packages /src/packages
RUN cd /src/dotnet-function-app && \
mkdir -p /home/site/wwwroot && \
dotnet restore --runtime linux-x64 && \
dotnet publish --configuration Release --runtime linux-x64 --self-contained true --output /home/site/wwwroot --no-restore

FROM mcr.microsoft.com/azure-functions/dotnet:3.0
ENV AzureWebJobsScriptRoot=/home/site/wwwroot \
AzureFunctionsJobHost__Logging__Console__IsEnabled=true \
COMPlus_EnableDiagnostics=0

## Run as non root user

WORKDIR /home/site/wwwroot

RUN addgroup --group app --gid 10001 \
&& useradd --uid 10001 --gid 10001 "app" \
&& chown app:app /home/site/wwwroot

USER app:app

COPY --from=installer-env ["/home/site/wwwroot", "/home/site/wwwroot"]
```
Please, note that the `COMPlus_EnableDiagnostics` environment variable MUST be set to `0` (either in the `Dockerfile` or in the K8S deployment manifest) otherwise, the function fails to even start with the following, cryptic error message:

```
Failed to create CoreCLR, HRESULT: 0x80004005
```

#### Related information

Provide any related information

* Programming language used : C#
* Bindings used: None

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.