Visualize Nvidia GPU Usage

You can use the tool nvtop (Linux only) to visualize the usage of Nvidia GPUs. However, it is only available on Linux and is not suitable for tracking and visualize the GPU usage in a long time period. Another simple approach to track and visualize the GPU usage is to dump GPU usage statistics into a CSV file using the following command

nvidia-smi --query-gpu=timestamp,name,pci.bus_id,driver_version,pstate,\
    pcie.link.gen.max,pcie.link.gen.current,temperature.gpu,utilization.gpu,\
    utilization.memory,memory.total,memory.free,memory.used --format=csv -l 5

and then visualize it using Python.

import pandas as pd

import hvplot
import hvplot.pandas

!curl -sSL www.legendu.net/media/ai/gpu_usage.csv -o gpu_usage.csv

df = pd.read_csv("gpu_usage.csv")
df.columns = [
    col.replace("[MiB]", "").replace("[%]", "").replace(".", "_").strip()
    for col in df.columns
]
df["timestamp"] = pd.to_datetime(df.timestamp)
df["memory_used"] = df["memory_used"].str.replace("MiB", "").astype(float)
df

df.hvplot.scatter("timestamp", "memory_used", size=0.5)