Ben Chuanlong Du's Blog

And let it direct your passion with reason.

Install Python Packages Behind Firewall

It is recommended that you use pip to install Python packages.

  1. If you don't already know the proxy in use (in your company), read the post Find out Proxy in Use to figure it out.

  2. Set proxy environment variables.

    set http_proxy=http://user:password@proxy_ip:port
set https_proxy=https://user …

Synchronization and Backup Solutions

Tips and Traps

  1. Almost all sychronization tools have problems with symbolic links. Most of them just ignore symbolic links. This makes sense as following symbolic links can causes serious disk problems. If not followed then symbolic links are likely broken on other device. The simple suggestion is do not place …

Docker for Nvidia GPU

Instruction on Using Nvidia GPU (CUDA) for Computing in Docker

  1. Install Nvidia cuda-drivers (or equivalent) on your Linux machine following instructions at CUDA Downloads. Notice that instead of installing cuda (using sudo apt-get install cuda), it is suggested that you install cuda-drivers only (using sudo apt-get install cuda-drivers). This is …

Visualize Nvidia GPU Usage

You can use the tool nvtop (Linux only) to visualize the usage of Nvidia GPUs. However, it is only available on Linux and is not suitable for tracking and visualize the GPU usage in a long time period. Another simple approach to track and visualize the GPU usage is to dump GPU usage statistics into a CSV file using the following command

Build Docker Images Using Kaniko

  1. Kaniko works differently from Docker. It runs inside a Docker container and detect and extract new layers to build Docker images. Since Kaniko manipulates the filesystem (layers) inside the Docker container, it can have unexpected side effect if not used carefully. For this reason, the developer team suggests users to …