-
Kaniko works differently from Docker. It runs inside a Docker container and detect and extract new layers to build Docker images. Since Kaniko manipulates the filesystem (layers) inside the Docker container, it can have unexpected side effect if not used carefully. For this reason, the developer team suggests users to …
Run Docker Containers Inside a Docker Container
You can run Docker containers inside a Docker container.
To allow this,
you have to pass the docker socks into the container
using the option -v /var/run/docker.sock:/var/run/docker.sock.
For more discussions,
please refer to
How To Run Docker in Docker Container [3 Easy Methods …
Packaging Python Dependencies for PySpark Using python-build-standalone
You can build a portable Python environment following steps below.
-
Install python-build-standalone.
-
Install Python packages using pip of the installed python-build-standalone distribution.
-
Pack the whole python-build-standalone directory into a compressed file, e.g.,
env.tar.gz.
The GitHub repo dclong/python-portable has good examples of building portable Python environments leveraging …
Useful Visual Studio Code Extensions
Places to Find Extensoins
Visual Studio Code Marketplace and Open VSX Registry are 2 places to find VSCode compatible extensions.
Install VSCode Extensions from Command-line
https://stackoverflow.com/questions/34286515/how-to-install-visual-studio-code-extensions-from-command-line/34339780#34339780
Install Code-Server Extensions from Command-line
If you install extension in Dockerfile using root,
the extensions are installed …
Tips on pex
Steps to Build a pex Environment File
-
Start a Python Docker image with the right version of Python interpreter installed. For example,
docker run -it -v $(pwd):/workdir python:3.5-buster /bin/bash -
Install pex.
pip3 install pex -
Build a pex environment file.
pex --python=python3 -v pyspark findspark -o …
Replace Single Quotes With Double Quotes in Python Code
There are 2 ways.
-
Format the Python code using black, which will automatically convert single quotes to double quotes when applicable. (Note that you can format the code again using yapf if you want the code to formatted by yapf finally.)
-
Use the tool myint/unify to help you.