It is suggested that you use python-build-standlone instead of conda-pack to build portable Python environments. Please refer to Packaging Python Dependencies for PySpark Using Python-Build-Standalone for more details.
All packages in a virtual environment must be managed by conda (rather than pip) so that it can be packe using conda-pack …
Start a Python Docker image with the right version of Python interpreter installed. For example,
docker run -it -v $(pwd):/workdir python:3.5-buster /bin/bash
Install pex.
pip3 install pex
Build a pex environment file.
pex --python=python3 -v pyspark findspark -o …By default,
Poetry create virtual environment in $HOME/.poetry for cahcing/sharing purpose.
However,
it is inconvenient if you use Docker-based web IDE for a Python project
as you will lose the created virtual environment once the Docker container is restarted.
In that situation,
you …