AI-Hypercomputer / AI-Hypercomputer/cloud-accelerator-diagnostics
[Bug] `tpu-info` can not access metrics
- 主要语言
- Python
- 星标
- 32
- 派生
- 15
- PR 合并指标
- 30 天内没有已合并 PR
描述
Minimal working code is [Here](https://gist.github.com/steveepreston/acd125a08214c631ba8389eb61a13798). Code follows [GoogleCloudPlatform example](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/official/training/tpuv5e_llama2_pytorch_finetuning_and_serving.ipynb)
Code run completed successfully on TPU VM v3-8, but while call `!tpu-info` at the end, it shows:
```
TPU Chips
┏━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━┓
┃ Chip ┃ Type ┃ Devices ┃ PID ┃
┡━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━┩
│ /dev/accel0 │ TPU v3 chip │ 2 │ 13 │
│ /dev/accel1 │ TPU v3 chip │ 2 │ 13 │
│ /dev/accel2 │ TPU v3 chip │ 2 │ 13 │
│ /dev/accel3 │ TPU v3 chip │ 2 │ 13 │
└─────────────┴─────────────┴─────────┴─────┘
Libtpu metrics unavailable. Is there a framework using the TPU? See https://github.com/google/cloud-accelerator-diagnostics/tree/main/tpu_info for more information
```
贡献指南
评估
这个 Issue 还没有评估数据。