Ben Chuanlong Du's Blog

And let it direct your passion with reason.

Read Tensorboard Logs

Using pandas.read_csv

This approaches requires you to have a running TensorBoard which is serving the data you want to read.

  1. Check the checkbox "Show data download links". See highlighted in the top-left corner of the screenshot below for an example.

  2. Select an experimentation whose you'd like to download. See highlighted in the bottom-right corner of the screenshot for an example.

  3. Right click on the CSV link and use the pop-up menu to copy the link. See highlighted in the bottom-right corner of the screenshot for an example.

  4. Now you can read the data into pandas using copied link. See the Python code below the screenshot for an example.

tensorboard

In [1]:
import pandas as pd
In [2]:
pd.read_csv(
    "http://10.0.0.185:6006/data/plugin/scalars/scalars?tag=Test+Loss&run=ofcp_resnet18_8&format=csv"
)
Out[2]:
Wall time Step Value
0 1.632517e+09 1 1.226414e-04
1 1.632517e+09 2 2.588446e-05
2 1.632517e+09 3 9.192328e-06
3 1.632517e+09 4 4.610128e-06
4 1.632517e+09 5 2.639660e-06
... ... ... ...
115 1.632529e+09 116 1.307412e-07
116 1.632529e+09 117 2.098449e-07
117 1.632529e+09 118 1.127275e-07
118 1.632529e+09 119 1.751949e-07
119 1.632529e+09 120 2.097909e-07

120 rows × 3 columns

tensorboard.backend.event_processing.event_accumulator.EventAccumulator

This approach has dependency on the Python library tensorboard, however, it does not need an running instance of TensorBoard. You just need to know the location of TensorBoard logs.

In [1]:
from tensorboard.backend.event_processing.event_accumulator import EventAccumulator
In [2]:
import pandas as pd
In [7]:
acc = EventAccumulator(
    "/workdir/ofcp_resnet18_8/events.out.tfevents.1632516849.jupyterhub-pytorch.21037.0"
)
In [9]:
acc = EventAccumulator(
    "/workdir/archives/ofcp_data/tensorboard/ofcp_resnet18_8/events.out.tfevents.1632516849.jupyterhub-pytorch.21037.0"
)
In [10]:
acc.Reload()
Out[10]:
<tensorboard.backend.event_processing.event_accumulator.EventAccumulator at 0x7fa4f00f3a30>
In [12]:
acc.Tags()
Out[12]:
{'images': [],
 'audio': [],
 'histograms': [],
 'scalars': ['Training Loss',
  'Training Accuracy',
  'Test Loss',
  'Test Accuracy'],
 'distributions': [],
 'tensors': [],
 'graph': False,
 'meta_graph': False,
 'run_metadata': []}
In [13]:
pd.DataFrame(acc.Scalars("Test Loss"))
Out[13]:
wall_time step value
0 1.632517e+09 1 1.226414e-04
1 1.632517e+09 2 2.588446e-05
2 1.632517e+09 3 9.192328e-06
3 1.632517e+09 4 4.610128e-06
4 1.632517e+09 5 2.639660e-06
... ... ... ...
115 1.632529e+09 116 1.307412e-07
116 1.632529e+09 117 2.098449e-07
117 1.632529e+09 118 1.127275e-07
118 1.632529e+09 119 1.751949e-07
119 1.632529e+09 120 2.097909e-07

120 rows × 3 columns

In [ ]:

Comments