Tips on Delta Lake

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Delta Table¶

convert to delta [db_name.]table_name [partitioned by ...] [vacuum [retain number hours]]

vaccum

describe history db_name.table_name

can select from historical snapshot can also rollback to a historical snapshot rollback is kind of dangerous as once rollback, commits after the rollback version are remove so that you cannot undo the rollback!

References¶

Delta Lake PySpark Examples

Delta Lake在eBay的实践：Spark SQL增删改查

Table Deletes, Updates, and Merges

Delta lake , ACID transactions for Apache Spark

Delta Lake quickstart

Table batch reads and writes

Table deletes, updates, and merges

Presto and Athena to Delta Lake integration