| GPU | model | busy | memory | power | temp | SM clock | PCIe in | PCIe out | experts on GPU | tok/s |
|---|---|---|---|---|---|---|---|---|---|---|
| waiting for data | ||||||||||
| run | mode | GPU store / pool | guesses | tok/s | L0 hit | L1 hit | L2 miss | SSD GB/s | RAM→GPU GB/s | longest layer wait |
|---|
API: /metrics (Prometheus) · /api/v1/summary · /api/v1/series · /api/v1/gpus · /api/v1/runs