ceph / ceph/ceph-nvmeof

Need to handle SPDK errors in update()

Open
#451 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
132
Forks
69
Avg merge
1d 7h
Merged PRs (30d)
47

Description

When we call SPDK during a CLI command, we return any error status to the user. The user can see the error and decide what to do next. But, when we run update() to sync the local state with the OMAP one and call the different SPDK functions these error statuses are getting lost. Nobody on the other side does anything with them. When the update() is over we will reset the local state to be equal to the OMAP one in any case. The result is we might get an error and not create or delete a resource. This means that our local state in the end of the update wouldn't reflect the true state and the local gateway will be out of sync from the other gateways and wouldn't even try to fix it as it looks at the fake local state. We should decide what to do in such cases. The easiest option is to crash the gateway. A new one will be started and we'll try to create all the resources from scratch. Another option is to quit the update() on such an error and flag it to not update the local state. This will make the gateway try to redo the actions in the next update as the local state will still be old. The problem here is that we did perform some of the actions so now we'll have to deal with cases of re-creating an existing resource or deleting a non-existent one. This complicates the processing considerably.

Contributor guide

Open the contributing guide

Research direction

Start by tracing update() and the SPDK functions it calls, focusing on where error statuses are discarded. Resolve the failure-handling behavior for partial updates, including how local state and subsequent retries should be treated; the issue does not name specific files or tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.