I countered the issue when using PySpark locally
(the issue can happen to a cluster as well).
It turned out to be caused by a misconfiguration of the environment variable JAVA_HOME
in Docker.
References
PySpark: Exception: Java gateway process exited before sending the driver its port number