kv/kvserver: TestGeneralOperationsWorkAsExpectedOnDeletedExternalSST failed

kv/kvserver.TestGeneralOperationsWorkAsExpectedOnDeletedExternalSST [failed](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Ci_TestsIbmCloudLinuxS390x_UnitTestsS390x/21476690?buildTab=log) with [artifacts](https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_Ci_TestsIbmCloudLinuxS390x_UnitTestsS390x/21476690?buildTab=artifacts#/) on master @ [8499e276da6fc483c164d1c99fc5f0c33a0e995f](https://github.com/cockroachdb/cockroach/commits/8499e276da6fc483c164d1c99fc5f0c33a0e995f):

```
| pkg/server/node.go:1943
| github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.func1
| pkg/rpc/context.go:822
| github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.ServerInterceptor.func13
| pkg/util/tracing/grpcinterceptor/grpc_interceptor.go:60
| github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func5
| pkg/rpc/context.go:933
| github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func3
| pkg/rpc/context.go:197
| github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func5
| pkg/rpc/context.go:933
| github.com/cockroachdb/cockroach/pkg/rpc.kvAuth.unaryInterceptor
| pkg/rpc/auth.go:107
| github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func5
| pkg/rpc/context.go:933
| github.com/cockroachdb/cockroach/pkg/rpc.gatewayRequestRecoveryInterceptor
| pkg/rpc/metrics.go:814
| github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func5
| pkg/rpc/context.go:933
| github.com/cockroachdb/cockroach/pkg/server.newGRPCServer.NewRequestMetricsInterceptor.func4
| pkg/rpc/metrics.go:600
| github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func5
| pkg/rpc/context.go:933
| github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func1.1
| pkg/rpc/context.go:154
| github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunTaskWithErr
| pkg/util/stop/stopper.go:358
| github.com/cockroachdb/cockroach/pkg/rpc.NewServerEx.func1
| pkg/rpc/context.go:152
| github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.chainUnaryServerInterceptors.bindUnaryServerInterceptorToHandler.func5
| pkg/rpc/context.go:933
| github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.func2
| pkg/rpc/context.go:832
| github.com/cockroachdb/cockroach/pkg/rpc.NewContext.ClientInterceptor.func13
| pkg/util/tracing/grpcinterceptor/grpc_interceptor.go:185
| github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.getChainUnaryInvoker.func4
| pkg/rpc/context.go:1017
| github.com/cockroachdb/cockroach/pkg/rpc.makeInternalClientAdapter.func3
| pkg/rpc/context.go:902
| github.com/cockroachdb/cockroach/pkg/rpc.internalClientAdapter.Batch
| pkg/rpc/context.go:1025
| github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*grpcTransport).sendBatch
| pkg/kv/kvclient/kvcoord/transport.go:229
| github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*grpcTransport).SendNext
| pkg/kv/kvclient/kvcoord/transport.go:191
Wraps: (2) split at key "e-15000" failed
Wraps: (3) pebble corruption error on store id:1, path:remote-nodelocal://1/external-files://file1.sst, remote:true, extra message: open /artifacts/tmp/_tmp/cface9c02edfdad8c648bb77dc94e533/TestGeneralOperationsWorkAsExpectedOnDeletedExternalSST2539937327/001/external-files/file1.sst: no such file or directory: nodelocal storage file does not exist: external_storage: file does not exist
Error types: (1) *errbase.opaqueWrapper (2) *errutil.withPrefix (3) *kvpb.PebbleCorruptionError
Test: TestGeneralOperationsWorkAsExpectedOnDeletedExternalSST/split_with_deleted_span_at_range_boundaries
--- FAIL: TestGeneralOperationsWorkAsExpectedOnDeletedExternalSST/split_with_deleted_span_at_range_boundaries (5.25s)
```
Help

See also: [How To Investigate a Go Test Failure \(internal\)](https://cockroachlabs.atlassian.net/l/c/HgfXfJgM)

/cc @cockroachlabs/kv-triage

[Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/bazci/githubpost/issues)

Jira issue: CRDB-67000

Start by reproducing kv/kvserver.TestGeneralOperationsWorkAsExpectedOnDeletedExternalSST, especially the split_with_deleted_span_at_range_boundaries case, using the linked TeamCity log and artifacts. Trace the reported failure through pkg/kv/kvclient/kvcoord/transport.go, pkg/rpc/context.go, and pkg/server/node.go. Done means the test no longer fails with the missing external file or Pebble corruption error.

Written by the indexing model from the issue text.