Ben Chuanlong Du's Blog

It is never too late to learn.

Make Your Model Training Reproducible in PyTorch

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

The PyTorch doc Reproducibility has very detailed instructions on how to make your model training reproducible. Basically, you need the following code.

torch.manual_seed(args.seed)
np.random.seed(args.seed)
random.seed(args.seed)

Notice that torch.manual_seed set seeds of RNGs on all devices (both CPU and GPUs). There is no need to make additional calls of torch.cuda.manual_seed or torch.manual_seed_all.

References

REPRODUCIBILITY

torch.manual_seed

What is manual_seed?

Difference between torch.manual_seed and torch.cuda.manual_seed

Comments