linode / linode/linode-blockstorage-csi-driver

pv failure on unexpected reboot

Open
#390 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Go
Stars
76
Forks
62
Avg merge
10h 58m
Merged PRs (30d)
10

Description

In case of single node cluster, pod finish by working after some time.

kubectl describe pod

  Warning  NodeNotReady            5m9s                   node-controller          Node is not ready
  Warning  FailedMount             5m                     kubelet                  MountVolume.MountDevice failed for volume "pvc-89cca58d07ec4a2c" : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name linodebs.csi.linode.com not found in the list of registered CSI drivers
  Warning  FailedMount             3m50s (x7 over 4m57s)  kubelet                  MountVolume.MountDevice failed for volume "pvc-89cca58d07ec4a2c" : rpc error: code = Internal desc = Unable to find device path out of attempted paths: [/dev/disk/by-id/linode-pvc89cca58d07ec4a2c /dev/disk/by-id/scsi-0Linode_Volume_pvc89cca58d07ec4a2c]
  Normal   SandboxChanged          2m46s                  kubelet                  Pod sandbox changed, it will be killed and re-created.
  Normal   Pulled                  2m46s                  kubelet                  Container image "nginx" already present on machine
  Normal   Created                 2m46s                  kubelet                  Created container nginx
  Normal   Started                 2m46s                  kubelet                  Started container nginx

In case of pod is rescheduled on another node, it loops not working:

  Warning  FailedMount             26m (x4 over 26m)  kubelet                  MountVolume.MountDevice failed for volume "pvc-93545880aa704102" : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name linodebs.csi.linode.com not found in the list of registered CSI drivers
  Warning  FailedMount             9s (x17 over 26m)  kubelet                  MountVolume.MountDevice failed for volume "pvc-93545880aa704102" : rpc error: code = Internal desc = Unable to find device path out of attempted paths: [/dev/disk/by-id/linode-pvc-93545880aa704102 /dev/disk/by-id/scsi-0Linode_Volume_pvc-93545880aa704102]

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the kubectl describe pod output and compare the single-node reboot case with the pod-rescheduled case. Trace CSI driver registration and the reported device paths for the Linode volume. Done means the pod can mount its volume and become healthy after both an unexpected reboot and rescheduling to another node.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.