Spark Issue: ViewFs: Cannot Initialize: Empty Mount Table in Config

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Symptoms¶

java.io.IOException: ViewFs: Cannot initialize: Empty Mount table in config for viewfs://cluster-name-ns02/

Possible Causes¶

As the error message says, viewfs://cluster-name-ns02 is not configured.

1. It is possible that the Spark cluster has just migrated to Router-based Federation (RBF) namenodes, but the Spark client is not updated correspondingly.

2. The HDFS path is not configured to be accessible.

Possible Solutions¶

1. Ask the Hadoop admin to update the Hadoop/Spark client (both the Hadoop binary and configuration files) if this is due to lacking of RBF compatibility.

2. Ask the Hadoop admin to configure viewfs://cluster-name-ns02 if the issue is due to misconfiguration.

3. Use a different HDFS path.