oracle / oracle/oci-cloud-controller-manager
Vol mount failed - iscsi: error logging in target: exit status 15
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 158
- Forks
- 108
- PR merge metrics
- No merged PRs in 30d
Description
Issue description:
On rebooting node, pod with pvc comes in unknown state and throws following error:
Pod describe o/p:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedMount 4m1s (x659 over 22h) kubelet MountVolume.MountDevice failed for volume "csi-b4786769-b306-40ac-9726-dc7aa313b780" : rpc error: code = Internal desc = iscsi: error logging in target: exit status 15
Environment details
CSI release =v1.33.0
K8 version = v1.35.0
OS release = "Ubuntu 24.04.3 LTS"
CSI Node error:
2026-02-06T10:34:54.428Z ERROR BV driver/bv_node.go:192 failed to log into the iSCSI target. {"volumeID": "ocid1.volume.oc1.eu-frankfurt-1.abtheljtbgocla7izk7phwdg3gjimtytpchwyr4chq24ukg7eyhsgca4baza", "stagingPath": "/var/lib/kubelet/plugins/kubernetes.io/csi/blockvolume.csi.oraclecloud.com/325b1e64dcdccc7ca1e75974ac73df635f1b000889017e241b8d8cdd7ba3e7cf/globalmount", "error": "iscsi: error logging in target: exit status 15"}
2026-02-06T10:34:54.428Z ERROR BV driver/driver.go:352 Failed to process gRPC request. {"error": "rpc error: code = Internal desc = iscsi: error logging in target: exit status 15", "method": "/csi.v1.Node/NodeStageVolume", "request": "{\"publish_context\":{\"attachment-type\":\"iscsi\",\"device\":\"/dev/oracleoci/oraclevdb\",\"iscci_iqn\":\"iqn.2015-12.com.oracleiaas:274e11de-bd7c-437f-a439-e9b4f6de6909\",\"iscsi_ip\":\"169.254.2.3\",\"iscsi_port\":\"3260\",\"multipathDevices\":\"\",\"multipathEnabled\":\"false\",\"needResize\":\"false\",\"newSize\":\"\",\"vpusPerGB\":\"10\"},\"staging_target_path\":\"/var/lib/kubelet/plugins/kubernetes.io/csi/blockvolume.csi.oraclecloud.com/325b1e64dcdccc7ca1e75974ac73df635f1b000889017e241b8d8cdd7ba3e7cf/globalmount\",\"volume_capability\":{\"access_mode\":{\"mode\":\"SINGLE_NODE_WRITER\"},\"mount\":{}},\"volume_context\":{\"attachment-type\":\"\",\"needResize\":\"false\",\"newSize\":\"\",\"storage.kubernetes.io/csiProvisionerIdentity\":\"1770228548244-4488-blockvolume.csi.oraclecloud.com\",\"vpusPerGB\":\"10\"},\"volume_id\":\"ocid1.volume.oc1.eu-frankfurt-1.abtheljtbgocla7izk7phwdg3gjimtytpchwyr4chq24ukg7eyhsgca4baza\"}"}
2026-02-06T10:34:57.802Z INFO FSS driver/driver.go:354 gRPC response is sent successfully. {"method": "/csi.v1.Node/NodeGetCapabilities", "response": "{\"capabilities\":[{\"rpc\":{\"type\":\"STAGE_UNSTAGE_VOLUME\"}}]}"}
2026-02-06T10:34:57.803Z INFO FSS driver/driver.go:354 gRPC response is sent successfully. {"method": "/csi.v1.Node/NodeGetCapabilities", "response": "{\"capabilities\":[{\"rpc\":{\"type\":\"STAGE_UNSTAGE_VOLUME\"}}]}"}
2026-02-06T10:34:57.804Z INFO FSS driver/driver.go:354 gRPC response is sent successfully. {"method": "/csi.v1.Node/NodeGetCapabilities", "response": "{\"capabilities\":[{\"rpc\":{\"type\":\"STAGE_UNSTAGE_VOLUME\"}}]}"}
2026-02-06T10:34:57.806Z DEBUG FSS driver/fss_node.go:62 volumeHandler : &{ocid1.filesystem.oc1.eu_frankfurt_1.aaaaaaaaaandin3ymzzgcllqojxwiotfouwwm4tbnzvwm5lsoqwtcllbmqwtgaaa 10.0.0.8 /csi-fss-083e7b39-9f3d-469f-8f59-68f96c25e5b8} {"volumeID": "ocid1.filesystem.oc1.eu_frankfurt_1.aaaaaaaaaandin3ymzzgcllqojxwiotfouwwm4tbnzvwm5lsoqwtcllbmqwtgaaa:10.0.0.8:/csi-fss-083e7b39-9f3d-469f-8f59-68f96c25e5b8"}
2026-02-06T10:34:57.806Z DEBUG FSS driver/fss_node.go:78 volume context: map[encryptInTransit:false storage.kubernetes.io/csiProvisionerIdentity:1770275984752-42-fss.csi.oraclecloud.com] {"volumeID": "ocid1.filesystem.oc1.eu_frankfurt_1.aaaaaaaaaandin3ymzzgcllqojxwiotfouwwm4tbnzvwm5lsoqwtcllbmqwtgaaa:10.0.0.8:/csi-fss-083e7b39-9f3d-469f-8f59-68f96c25e5b8"}
2026-02-06T10:34:57.816Z DEBUG FSS driver/fss_node.go:130 Trying to stage. {"volumeID": "ocid1.filesystem.oc1.eu_frankfurt_1.aaaaaaaaaandin3ymzzgcllqojxwiotfouwwm4tbnzvwm5lsoqwtcllbmqwtgaaa:10.0.0.8:/csi-fss-083e7b39-9f3d-469f-8f59-68f96c25e5b8"}
2026-02-06T10:34:57.816Z INFO FSS driver/fss_node.go:149 Stage started. {"volumeID": "ocid1.filesystem.oc1.eu_frankfurt_1.aaaaaaaaaandin3ymzzgcllqojxwiotfouwwm4tbnzvwm5lsoqwtcllbmqwtgaaa:10.0.0.8:/csi-fss-083e7b39-9f3d-469f-8f59-68f96c25e5b8"}
--
At OS layer, it seems the device is already logged in however CSI Driver also tried to do and fails
ubuntu@instance-20260202-1239:/var/log$ sudo iscsiadm -m session
tcp: [1] 169.254.0.2:3260,1 iqn.2015-02.oracle.boot:uefi (non-flash)
tcp: [2] 169.254.2.3:3260,1 iqn.2015-12.com.oracleiaas:274e11de-bd7c-437f-a439-e9b4f6de6909 (non-flash)
If I logout from session using iscsiadm command at OS layer and restart the pod, then volume is mounted successfully. This manual workaround is not feasible for production environment.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with driver/bv_node.go:192 and the NodeStageVolume handling reported through driver/driver.go:352, then inspect the attached csi-oci-node.tar.gz logs. Compare the CSI node's iSCSI login attempt with the existing iscsiadm sessions after reboot. Done means the volume mounts without manually logging out the existing session, with the behavior verified in the reported environment.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100