adobe / adobe/rules_gitops

Race condition in it_sidecar

Abierto
#130 0 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Starlark
Estrellas
209
Forks
45
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

@harjsing7 reported a race condition in a sidecar on pod removal.
```
2023-03-22T05:44:02Z | INFO | 2023/03/22 05:44:02 panic: close of closed channel
2023-03-22T05:44:02Z | INFO | 2023/03/22 05:44:02
2023-03-22T05:44:02Z | INFO | 2023/03/22 05:44:02 goroutine 108 [running]:
2023-03-22T05:44:02Z | INFO | 2023/03/22 05:44:02 github.com/adobe/rules_gitops/testing/it_sidecar/stern.(*Tail).Start.func2()
2023-03-22T05:44:02Z | INFO | 2023/03/22 05:44:02 external/com_adobe_rules_gitops/testing/it_sidecar/stern/tail.go:86 +0x45
2023-03-22T05:44:02Z | INFO | 2023/03/22 05:44:02 created by github.com/adobe/rules_gitops/testing/it_sidecar/stern.(*Tail).Start
2023-03-22T05:44:02Z | INFO | 2023/03/22 05:44:02 external/com_adobe_rules_gitops/testing/it_sidecar/stern/tail.go:84 +0x12b
2023-03-22T05:44:02Z | INFO | 2023/03/22 05:44:02 exit status 2
```
The closing of the channel can happen in two spots:
https://github.com/adobe/rules_gitops/blob/eafc9c3e0eac03f9487abb22e3b735c47ff1a043/testing/it_sidecar/stern/tail.go#L86
https://github.com/adobe/rules_gitops/blob/eafc9c3e0eac03f9487abb22e3b735c47ff1a043/testing/it_sidecar/stern/main.go#L56

We need to handle this situation properly, using cancellable context would be appropriate here
Also, we should add a warning in a log on pod deletion, since it is typically an indication of a problem in the test - like pod crashlooping or even early test failure (configurable)

Guía de contribución

Abrir la guía de contribución

Línea de trabajo

Start with testing/it_sidecar/stern/tail.go at line 86 and testing/it_sidecar/stern/main.go at line 56, where the channel is closed during pod removal. Trace the sidecar shutdown path and the existing pod-deletion logging, then run the relevant integration test. Done means pod removal no longer panics from closing the channel and emits a warning when deletion indicates a possible test problem.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
go, kubernetes
Área
devops, testing-qa
Tipo de issue
Error
Dificultad
4/5
Tiempo estimado
3-5 días
Estado de actividad
Estancado
Claridad
Bastante claro
Aptitud para principiantes
42/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.