Ben Chuanlong Du's Blog

And let it direct your passion with reason.

Spark Issue: Namespace Quota Is Exceeded

Symptom

Caused by: org.apache.hadoop.hdfs.protocol.NSQuotaExceededException: The NameSpace quota (directories and files) of directory /user/user_name is exceeded: quota=163840 file count=163841

Cause

The namespace quota of the directory /user/user_name is execeeded.

Solutions

  1. Remove non-needed files from the directory /user/user_name to release some namespace quota.

  2. As a long-term solution, you can also try to apply for more resource for the HDFS directory /user/user_name.

Comments