Tips and Traps¶
- HDFS table might contain invalid data (I'm not clear about the reasons at this time) with respct to the column types (e.g., Date and Timestamp). This will cause issues when Spark tries to load the data. For more discussions, please refer to Unrecognized column type:TIMESTAMP_TYP.
datetime.datetimeordatetime.date
Format Date and Time in Python
Date/Time Format¶
Please refer to https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes for a complete list of formats.