It is suggested that you always pass a list of columns to the parameter on even if there's only one column for joining.

None in a pandas DataFrame is converted to NaN instead of null!

Spark allows using following join types:

• inner (default)
• cross
• outer
• full, fullouter, full_outer
• left, leftouter, left_outer
• right, rightouter, right_outer
• semi, leftsemi, left_semi
• anti, leftanti, left_anti

Select only needed columns before joining.

Rename joining column names to be identical (if different) before joining.

TABLESAMPLE must be immedidately after a table name.

The WHERE clause in the following SQL query runs after TABLESAMPLE.

 SELECT 
     *
 FROM 
     table_name 
 TABLESAMPLE (10 PERCENT) 
 WHERE 
     id = 1

If you want to run a WHERE