Note: For Vim users, it is appealing to make the Caps Lock function like Escape. It is suggested that you make Caps Lock an additonal Escape rather than swap them. This avoid inconsistent key mapping issues when you work remotely via VNC, NoMachine, etc.

Desktop Environment

Some desktop environment (e …

Note: Cron is great for simple scheduling requests. For complicated scheduling with many dependencies you probably want to go with AirFlow .

Installation

Cron is usually installed by default. It not (e.g., if you are using a Docker image), you can install it using the following command (on Ubuntu).

wajig …

Installing GNU tools is a way to try to get Linux command experience in macOS. However, not every Linux command has an identical-experience version in macOS. ** It is suggested that you use a Linux virtual machine if you really want to have Linux experience in macOS. ** Notice that some hardwares …

1. There are 2 popular Python libraries tomlkit and toml for parsing TOML formatted files in Python. tomlkit is preferred to toml as it is more flexible and style-preserving.

2. A TOML file always interpret a key (even a bare ASCII integer) as string. For this reason, a dict with numerical keys …

Note: The tips here are for Debian-based Linux distributions (Debian, Ubuntu, LinuxMint, Pop!_OS, AntiX, etc.).

Debian Specific

1. If you installed Debian without a desktop environment, and wants to install a customized desktop environment now, make sure to install a DE manager (e.g., gdm, lightdm, etc.) as well. without …

1. PySpark 2.4 and older does not support Python 3.8. You have to use Python 3.7 with PySpark 2.4 or older.

2. It can be extremely helpful to run a PySpark application locally to detect possible issues before submitting it to the Spark cluster.

   #!/usr/bin/env bash …