Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!
Symptom¶
Exception in thread “main” java.lang.IllegalArgumentException: System memory 466092032 must be at least 471859200. Please increase heap size using the --driver-memory option or spark.driver.memory in Spark configuration.
Causes¶
Not enough heap size (dirver memory).
Solutions¶
Increase heap size using the --driver-memory option or spark.driver.memory in Spark configuration.