Comments¶

1. It is suggested that you always pass a list of columns to the parameter on even if there's only one column for joining.

2. None in a pandas DataFrame is converted to NaN instead of null!

3. Spark allows using following join types:

   • inner (default)
   • cross
   • outer
   • full, fullouter, full_outer
   • left, leftouter, left_outer
   • right, rightouter, right_outer
   • semi, leftsemi, left_semi
   • anti, leftanti, left_anti

Online Regular Expression Tester

1. The Python module re automatically compiles a plain/text pattern using re.compile and caches it, so there's not much benefit to compile plain/text patterns by yourself.

2. Some regular expression patterns are defined using a single leading backslash, e.g., \s, \b, etc. However, since special characters (e.g., \) need to be escaped in strings in most programming languages, you will need the string "\\s"

Tips & Traps¶

1. Please refer to Parse TOML File in Python for general tips on parsing TOML in Python.

Installatoion¶

Tips and Traps¶

1. Select only needed columns before joining.

2. Rename joining column names to be identical (if different) before joining.