canonical / canonical/microceph

Increase disk space on my primary system disk on a node, it looks like the snap services cannot start

Open
#281 21 comments 0 reactions 1 assignee Claimed by @UtkarshBhatthere View on GitHub
Dominant language
Go
Stars
396
Forks
74
Avg merge
2d 20h
Merged PRs (30d)
7

Description

I have a microceph question. I setup a 3 node cluster, got them all linked up running microk8s and microceph. I then started adding all my deployments and I started to get a health warning about low drive space. After some investigation turns out the disk (primary) that is running kube and the system stuff was low. I doubled the size of this disk (using my suppliers dash setup) and rebooted the now. now the space is there, the warning has gone. Microk8s is healthy and see's all nodes up. I can see all the nodes in microceph status, however the node I added the disk space on is now out of quorum. I tried running 'sudo systemctl restart ceph-mon@' but I got a unit not found error.

![image](https://github.com/canonical/microceph/assets/55986138/1bfc17f2-8a46-4c47-abce-4cc048066ef9)

snap service status:

snap.microceph.mon.service - Service for snap application microceph.mon
Loaded: loaded (/etc/systemd/system/snap.microceph.mon.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Wed 2024-01-03 13:00:24 UTC; 8min ago
Process: 1457999 ExecStart=/usr/bin/snap run microceph.mon (code=exited, status=1/FAILURE)
Main PID: 1457999 (code=exited, status=1/FAILURE)
CPU: 53ms

Jan 03 13:00:24 milan7763-14616 systemd[1]: snap.microceph.mon.service: Scheduled restart job, restart counter is at 5.
Jan 03 13:00:24 milan7763-14616 systemd[1]: Stopped Service for snap application microceph.mon.
Jan 03 13:00:24 milan7763-14616 systemd[1]: snap.microceph.mon.service: Start request repeated too quickly.
Jan 03 13:00:24 milan7763-14616 systemd[1]: snap.microceph.mon.service: Failed with result 'exit-code'.
Jan 03 13:00:24 milan7763-14616 systemd[
[journalctl -u snap microceph logs.txt](https://github.com/canonical/microceph/files/13819525/journalctl.-u.snap.microceph.logs.txt)
1]: Failed to start Service for snap application microceph.mon.
[microceph common logs.txt](https://github.com/canonical/microceph/files/13819526/microceph.common.logs.txt)

Could this be a networking issue to the primary node?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.