Sometime when you install softwares in Linux, you get an error message saying that you do not have permission while you used sudo or the root account. This probably means that you do not have full access to some installation files. An easy way to solve this problem is to …

Symptom 1

/bin/sh: hdfs: command not found

Possible Causes of Symptom 1

The command hdfs is not on the search path.

Possible Solutions to Symptom 1

1. Use the full path to the command.
2. Configure the environment variable PATH before you use the command.
3. Find other alternatives to the command …

Symptom

Caused by: org.apache.hadoop.hdfs.protocol.NSQuotaExceededException: The NameSpace quota (directories and files) of directory /user/user_name is exceeded: quota=163840 file count=163841

Cause

The namespace quota of the directory /user/user_name is execeeded.

Solutions

1. Remove non-needed files from the directory /user/user_name to release some namespace …

If you use Rust with Spark/PySpark and there are issues in the Rust code, you might get Rust panic error messages.

Symptom

Error: b"thread 'main' panicked at 'index out of bounds: the len is 15 but the index is 15', src/game.rs:131:39\nnote: run with …

Symptom

java.lang.RuntimeException: Unsupported literal type class java.util.ArrayList [1]

Possible Causes

This happens in PySpark when a Python list is provide where a scalar is required. Assuming id0 is an integer column in the DataFrame df, the following code throws the above error.

v = [1, 2, 3 …