Pulling/pushing Docker images is extremely slow in China. There are a few ways to speed up docker pulling/pushing in China.

Using the Official China Mirror

Now the best way is to use the offical China mirror. Please refer to Docker 中国官方镜像加速 …

__file__ is the path of the Python script. Note that if you make a sybolic link to a Python script and run the symbolic link, then __file__ is the path of the symbolic link. Of course, you can use os.path.realpath to get real path of files.

pathlib.Path …

1. Break down the work into smaller modules and develop pipelines (consists of module) for the work. Be sure to save (important) intermediate results so that you can resume failed modules withouting reruning succeeded ones.

2. Manage your project in GitHub and use issues to manage tasks to do and their priorities …

On the Same Machine

If the two directories are on the same machine, you can use either colordiff (preferred over diff) or git diff to find the differences between them.

colordiff -qr dir_1 dir_2

git diff --no-index dir_1 dir_2

On Different Machines

It is a little bit tricky when the …

List Files Sorted by Time

You can list files sorted by time (newest first) using the -t option. Notice that the -t option is also support by hdfs dfs -ls.

ls -lht

Ignore Files

1. You have to either enclose the pattern in quotes or escape the wildcard in patterns.

2. Equivalent …

Here is a good article on Git Behind a Proxy or Firewall.

1. If you do not already know the proxy in use (in your office), read the post Find out Proxy in Use to figure the proxy (in your office).

2. Run the following commands in terminal or Windows command prompt …