Azure / Azure/iotedge

Security daemon could not start successfully: Could not start management service

Open
#5,933 8 comments 0 reactions 1 assignee Claimed by @bilalsellak View on GitHub
1.0.9 customer-reported iotedge no-issue-activity
Dominant language
C#
Stars
1.5k
Forks
473
Avg merge
2d 17h
Merged PRs (30d)
7

Description

## Expected Behavior
edgeAgent, edgeHub and other custom modules should be started by IoT Edge service.

## Current Behavior
Security daemon failed to start up and module containers are not being created.

## Steps to Reproduce
Not sure how to replicate this.

## Context (Environment)

### Output of `iotedge check`

Click here

```
Configuration checks
--------------------
√ config.yaml is well-formed - OK
‼ config.yaml has well-formed connection string - Warning
Device not configured with manual provisioning, in this configuration 'iotedge check' is not able to discover the device's backing IoT Hub.
To run connectivity checks in this configuration please specify the backing IoT Hub name using --iothub-hostname switch if you have that information.
If no hostname is provided, all hub connectivity tests will be skipped.
√ container engine is installed and functional - OK
√ config.yaml has correct hostname - OK
× config.yaml has correct URIs for daemon mgmt endpoint - Error
Error: could not execute list-modules request: an IO error occurred: Connection reset by peer (os error 104)
‼ latest security daemon - Warning
Installed IoT Edge daemon has version 1.0.9 but 1.1.8 is the latest stable version available.
Please see https://aka.ms/iotedge-update-runtime for update instructions.
√ host time is close to real time - OK
√ container time is close to host time - OK
√ DNS server - OK
√ production readiness: certificates - OK
√ production readiness: container engine - OK
‼ production readiness: logs policy - Warning
Container engine is not configured to rotate module logs which may cause it run out of disk space.
Please see https://aka.ms/iotedge-prod-checklist-logs for best practices.
You can ignore this warning if you are setting log policy per module in the Edge deployment.
× production readiness: Edge Agent's storage directory is persisted on the host filesystem - Error
Could not check current state of edgeAgent container
× production readiness: Edge Hub's storage directory is persisted on the host filesystem - Error
Could not check current state of edgeHub container

Connectivity checks
-------------------
√ host can connect to and perform TLS handshake with DPS endpoint - OK

9 check(s) succeeded.
3 check(s) raised warnings. Re-run with --verbose for more details.
3 check(s) raised errors. Re-run with --verbose for more details.
9 check(s) were skipped due to errors from other checks. Re-run with --verbose for more details.
```

### Device Information
* Host OS [e.g. Ubuntu 18.04, Windows Server IoT 2019]: Debian 9
* Architecture [e.g. amd64, arm32, arm64]: armhf
* Container OS [e.g. Linux containers, Windows containers]: Linux

### Runtime Versions
* aziot-edged [run `iotedge version`]: 1.0.9
* Edge Agent [image tag (e.g. 1.0.0)]: 1.0.9
* Edge Hub [image tag (e.g. 1.0.0)]: 1.0.9
* Docker/Moby [run `docker version`]: 3.0.5

Note: when using Windows containers on Windows, run `docker -H npipe:////./pipe/iotedge_moby_engine version` instead

## Logs

aziot-edged logs

```
Dec 14 11:53:10 Moxa systemd[1]: Started Azure IoT Edge daemon.
Dec 14 11:53:10 Moxa iotedged[25313]: 2021-12-14T19:53:10Z [INFO] - Starting Azure IoT Edge Security Daemon
Dec 14 11:53:10 Moxa iotedged[25313]: 2021-12-14T19:53:10Z [INFO] - Version - 1.0.9
Dec 14 11:53:10 Moxa iotedged[25313]: 2021-12-14T19:53:10Z [INFO] - Using config file: /etc/iotedge/config.yaml
Dec 14 11:53:10 Moxa iotedged[25313]: 2021-12-14T19:53:10Z [INFO] - Configuring /var/lib/iotedge as the home directory.
Dec 14 11:53:10 Moxa iotedged[25313]: 2021-12-14T19:53:10Z [INFO] - Configuring certificates...
Dec 14 11:53:10 Moxa iotedged[25313]: 2021-12-14T19:53:10Z [INFO] - Configuring the Device CA certificate using "/var/thingspro/apps/cloud/data/setting/azureiotedge/enroll/iotEdge_moxa.crt".
Dec 14 11:53:10 Moxa iotedged[25313]: 2021-12-14T19:53:10Z [INFO] - Configuring the Device private key using "/var/thingspro/apps/cloud/data/setting/azureiotedge/enroll/iotEdge_moxa.key".
Dec 14 11:53:10 Moxa iotedged[25313]: 2021-12-14T19:53:10Z [INFO] - Configuring the trusted CA certificates using "/var/thingspro/apps/cloud/data/setting/azureiotedge/enroll/RootCA_moxa.crt".
Dec 14 11:53:10 Moxa iotedged[25313]: 2021-12-14T19:53:10Z [INFO] - Finished configuring provisioning environment variables and certificates.
Dec 14 11:53:10 Moxa iotedged[25313]: 2021-12-14T19:53:10Z [INFO] - Initializing hsm...
Dec 14 11:53:10 Moxa iotedged[25313]: 2021-12-14T19:53:10Z [INFO] - Finished initializing hsm.
Dec 14 11:53:10 Moxa iotedged[25313]: 2021-12-14T19:53:10Z [INFO] - Provisioning edge device...
Dec 14 11:53:10 Moxa iotedged[25313]: 2021-12-14T19:53:10Z [INFO] - Starting provisioning edge device via TPM...
Dec 14 11:53:11 Moxa iotedged[25313]: 2021-12-14T19:53:11Z [INFO] - Starting DPS registration with scope_id "0ne00236C76", registration_id "taiib1028226"
Dec 14 11:53:24 Moxa iotedged[25313]: 2021-12-14T19:53:24Z [INFO] - DPS registration assigned device "taiib1028226" in hub "pf-p314-iot-iothub-prod.azure-devices.net"
Dec 14 11:53:24 Moxa iotedged[25313]: 2021-12-14T19:53:24Z [INFO] - Finished provisioning edge device.
Dec 14 11:53:24 Moxa iotedged[25313]: 2021-12-14T19:53:24Z [INFO] - Initializing the module runtime...
Dec 14 11:53:24 Moxa iotedged[25313]: 2021-12-14T19:53:24Z [INFO] - Initializing module runtime...
Dec 14 11:53:24 Moxa iotedged[25313]: 2021-12-14T19:53:24Z [INFO] - Using runtime network id azure-iot-edge
Dec 14 11:53:24 Moxa iotedged[25313]: 2021-12-14T19:53:24Z [INFO] - Successfully initialized module runtime
Dec 14 11:53:24 Moxa iotedged[25313]: 2021-12-14T19:53:24Z [INFO] - Finished initializing the module runtime.
Dec 14 11:53:24 Moxa iotedged[25313]: 2021-12-14T19:53:24Z [INFO] - Reprovisioning status InitialAssignment will trigger reconfiguration of modules.
Dec 14 11:53:24 Moxa iotedged[25313]: 2021-12-14T19:53:24Z [INFO] - Detecting if configuration file has changed...
Dec 14 11:53:24 Moxa iotedged[25313]: 2021-12-14T19:53:24Z [INFO] - No change to configuration file detected.
Dec 14 11:53:24 Moxa iotedged[25313]: 2021-12-14T19:53:24Z [INFO] - Edge issuer CA expiration date: 2037-07-07T01:07:06Z
Dec 14 11:53:24 Moxa iotedged[25313]: 2021-12-14T19:53:24Z [INFO] - Obtaining workload CA succeeded.
Dec 14 11:53:24 Moxa iotedged[25313]: 2021-12-14T19:53:24Z [INFO] - Starting management API...
Dec 14 11:53:24 Moxa iotedged[25313]: 2021-12-14T19:53:24Z [INFO] - Starting workload API...
Dec 14 11:53:24 Moxa iotedged[25313]: 2021-12-14T19:53:24Z [INFO] - Starting watchdog with 60 second frequency...
Dec 14 11:53:24 Moxa iotedged[25313]: 2021-12-14T19:53:24Z [ERR!] - The daemon could not start up successfully: Could not start management service
Dec 14 11:53:24 Moxa iotedged[25313]: 2021-12-14T19:53:24Z [ERR!] - caused by: An error occurred for path /var/run/iotedge/mgmt.sock
Dec 14 11:53:24 Moxa iotedged[25313]: 2021-12-14T19:53:24Z [ERR!] - caused by: Permission denied (os error 13)
Dec 14 11:53:24 Moxa systemd[1]: iotedge.service: Main process exited, code=exited, status=1/FAILURE
Dec 14 11:53:24 Moxa systemd[1]: iotedge.service: Unit entered failed state.
Dec 14 11:53:24 Moxa systemd[1]: iotedge.service: Failed with result 'exit-code'.
Dec 14 11:53:25 Moxa systemd[1]: iotedge.service: Service hold-off time over, scheduling restart.
Dec 14 11:53:25 Moxa systemd[1]: Stopped Azure IoT Edge daemon.
```

## Additional Information
Permission of /var/run/iotedge/mgmt.sock
```
# ls -alh /var/run/iotedge/
total 0
drwxr-xr-x 2 root root 80 Dec 14 12:08 .
drwxr-xr-x 20 root root 740 Dec 14 11:53 ..
srw-rw---- 1 iotedge iotedge 0 Dec 14 12:08 mgmt.sock
srw-rw-rw- 1 iotedge iotedge 0 Dec 14 12:08 workload.sock
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.