Persist vs Checkpoint¶

Spark Internals - 6-CacheAndCheckpoint.md has a good explanation of persist vs checkpoint.

1. Persist/Cache in Spark is lazy and doesn't truncate the lineage while checkpoint is eager (by default) and truncates the lineage.

2. Generally speaking, DataFrame.persist has a better performance than DataFrame.checkpoint. However, DataFrame.checkpoint is more robust and is preferred in any of the following situations.