Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
Symptoms¶
java.lang.IllegalArgumentException: Wrong FS: hdfs://..., expected: viewfs://...
Possible Causes¶
The Spark cluster has migrated to Router-based Federation (RBF) namenodes,
and viewfs:// (instead of hdfs://) is required to access HDFS paths.
Possible Solutions¶
Use
viewfs://instead ofhdfs://.Use relative HDFS paths (without
viewfs://orhdfs://prefixes).