AI-Hypercomputer / AI-Hypercomputer/cloud-accelerator-diagnostics
[Bug] `tpu-info` can not access metrics
- Linguagem predominante
- Python
- Estrelas
- 32
- Forks
- 15
- Métricas de merge de PRs
- Nenhum PR com merge em 30d
Descrição
Minimal working code is [Here](https://gist.github.com/steveepreston/acd125a08214c631ba8389eb61a13798). Code follows [GoogleCloudPlatform example](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/training/tpuv5e_llama2_pytorch_finetuning_and_serving.ipynb)
Code run completed successfully on TPU VM v3-8, but while call `!tpu-info` at the end, it shows:
```
TPU Chips
┏━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━┓
┃ Chip ┃ Type ┃ Devices ┃ PID ┃
┡━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━┩
│ /dev/accel0 │ TPU v3 chip │ 2 │ 13 │
│ /dev/accel1 │ TPU v3 chip │ 2 │ 13 │
│ /dev/accel2 │ TPU v3 chip │ 2 │ 13 │
│ /dev/accel3 │ TPU v3 chip │ 2 │ 13 │
└─────────────┴─────────────┴─────────┴─────┘
Libtpu metrics unavailable. Is there a framework using the TPU? See https://github.com/google/cloud-accelerator-diagnostics/tree/main/tpu_info for more information
```
Guia de contribuição
Avaliação
Esta issue ainda não foi avaliada.