www.HadoopExam.com

HadoopExam Learning Resources

CCD-410 Certifcation CCA-500 Hadoop Administrator Exam HBase Certifcation CCB-400 Data Science Certifcation Hadoop Training with Hands On Lab Hadoop Package Deal

--jars works in "yarn-client" but not "yarn-cluster" mode, why?

Hadoop version: CDH 5.4.

 
We need to connect to HBase, thus need extra "/opt/cloudera/parcels/CDH/lib/hbase/lib/htrace-core-3.1.0-incubating.jar" dependency.
 
It works in yarn-client mode:
"spark-submit --class xxx.xxx.MyApp --master yarn-client --num-executors 10 --executor-memory 10g --jars /opt/cloudera/parcels/CDH/lib/hbase/lib/htrace-core-3.1.0-incubating.jar my-app.jar /input /output"
 
However, if we change "yarn-client" to "yarn-cluster', it throws an ClassNotFoundException (actually the class exists in htrace-core-3.1.0-incubating.jar):
 
Caused by: java.lang.NoClassDefFoundError: org/apache/htrace/Trace
	at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:218)
	at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:481)
	at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:65)
	at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:86)
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:850)
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:635)
	... 21 more
Caused by: java.lang.ClassNotFoundException: org.apache.htrace.Trace
	at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
	at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
 
Why --jars doesn't work in yarn-cluster mode? How to add extra dependency in "yarn-cluster" mode?

 
The issue you're running into is because most HBase classes are in the system class path, while jars added with "--jars" are only visible to the application class loader created by Spark. So classes in the system class path cannot see them.

You can work around this by setting "--driver-classpath /opt/.../htrace-core-3.1.0-incubating.jar" and "--conf spark.executor.extraClassPath=
/opt/.../htrace-core-3.1.0-incubating.jar" in your spark-submit command line. (You can also add those configs to your spark-defaults.conf to avoid having to type them all the time; and don't forget to include any other jars that might be needed.)

Add comment


Security code
Refresh

You are here: Home Question & Answer Hadoop Questions --jars works in "yarn-client" but not "yarn-cluster" mode, why?