Angel 3.0.1-hadoop3, hadoop3.1.1, angel-submit on yarn Failed for Hadoop NoClassDefFoundError

**error log:**
name: Bug report/Feature request/Question
2022-01-12 11:01:53,433 FATAL [main] com.tencent.angel.master.AngelApplicationMaster: Error starting AppMaster
java.lang.NoClassDefFoundError: org/apache/hadoop/mapreduce/lib/input/CombineTextInputFormat
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:756)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:468)
at java.net.URLClassLoader.access$100(URLClassLoader.java:74)
at java.net.URLClassLoader$1.run(URLClassLoader.java:369)
at java.net.URLClassLoader$1.run(URLClassLoader.java:363)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:362)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at com.tencent.angel.conf.AngelConf.(AngelConf.java:123)
at com.tencent.angel.master.AngelApplicationMaster$RunningAppContext.getRunningMode(AngelApplicationMaster.java:387)
at com.tencent.angel.master.AngelApplicationMaster.initAndStart(AngelApplicationMaster.java:779)
at com.tencent.angel.master.AngelApplicationMaster$1.run(AngelApplicationMaster.java:673)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at com.tencent.angel.master.AngelApplicationMaster.main(AngelApplicationMaster.java:671)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapreduce.lib.input.CombineTextInputFormat
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 20 more need this feature. if you required feature is big please connect us email list.

**Angel-submit add hadoop env:**
if [ `command -v hadoop` ];then
export HADOOP_CLASSPATH=`hadoop classpath`
else
echo "hadoop command not found in path!"
fi
HADOOP_HOME="/usr/hdp/3.1.4.0-315/hadoop"
if [ "${HADOOP_HOME}" != "" ]; then
echo "HADOOP_HOME is set"
DEFAULT_LIBEXEC_DIR="${HADOOP_HOME}"/libexec
HADOOP_LIBEXEC_DIR=${HADOOP_LIBEXEC_DIR:-$DEFAULT_LIBEXEC_DIR}
. ${HADOOP_LIBEXEC_DIR}/hadoop-config.sh
fi

**runtime log:**
HADOOP_HOME is set
WARNING: DEFAULT_LIBEXEC_DIR ignored. It has been replaced by HADOOP_DEFAULT_LIBEXEC_DIR.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/hdp/3.1.4.0-315/hadoop/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/root/angel-3.0.1-bin/lib/slf4j-log4j12-1.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
22/01/12 11:20:32 INFO utils.AngelRunJar: angelHomePath conf path=/root/angel-3.0.1-bin/bin/..//conf/angel-site.xml
22/01/12 11:20:32 INFO utils.AngelRunJar: load system config file success
22/01/12 11:20:32 INFO utils.AngelRunJar: for libJars..../root/angel-3.0.1-bin/bin/..//lib/angel-math-0.1.1.jar,/root/angel-3.0.1-bin/bin/..//lib/angel-format-0.1.1.jar,/root/angel-3.0.1-bin/bin/..//lib/angel-mlcore-0.1.2.jar,/root/angel-3.0.1-bin/bin/..//lib/jniloader-1.1.jar,/root/angel-3.0.1-bin/bin/..//lib/native_system-java-1.1.jar,/root/angel-3.0.1-bin/bin/..//lib/arpack_combined_all-0.1.jar,/root/angel-3.0.1-bin/bin/..//lib/all-1.1.2.pom,/root/angel-3.0.1-bin/bin/..//lib/core-1.1.2.jar,/root/angel-3.0.1-bin/bin/..//lib/netlib-native_ref-linux-armhf-1.1-natives.jar,/root/angel-3.0.1-bin/bin/..//lib/netlib-native_ref-linux-i686-1.1-natives.jar,/root/angel-3.0.1-bin/bin/..//lib/netlib-native_ref-linux-x86_64-1.1-natives.jar,/root/angel-3.0.1-bin/bin/..//lib/netlib-native_system-linux-armhf-1.1-natives.jar,/root/angel-3.0.1-bin/bin/..//lib/netlib-native_system-linux-i686-1.1-natives.jar,/root/angel-3.0.1-bin/bin/..//lib/netlib-native_system-linux-x86_64-1.1-natives.jar,/root/angel-3.0.1-bin/bin/..//lib/jackson-annotations-2.6.5.jar,/root/angel-3.0.1-bin/bin/..//lib/jackson-core-2.7.7.jar,/root/angel-3.0.1-bin/bin/..//lib/jackson-core-asl-1.9.13.jar,/root/angel-3.0.1-bin/bin/..//lib/jackson-databind-2.7.7.jar,/root/angel-3.0.1-bin/bin/..//lib/jackson-jaxrs-1.9.13.jar,/root/angel-3.0.1-bin/bin/..//lib/jackson-mapper-asl-1.9.13.jar,/root/angel-3.0.1-bin/bin/..//lib/jackson-module-paranamer-2.6.5.jar,/root/angel-3.0.1-bin/bin/..//lib/jackson-module-scala_2.11-2.6.5.jar,/root/angel-3.0.1-bin/bin/..//lib/jackson-xc-1.9.13.jar,/root/angel-3.0.1-bin/bin/..//lib/json4s-ast_2.11-3.2.11.jar,/root/angel-3.0.1-bin/bin/..//lib/json4s-core_2.11-3.2.11.jar,/root/angel-3.0.1-bin/bin/..//lib/json4s-jackson_2.11-3.2.11.jar,/root/angel-3.0.1-bin/bin/..//lib/netty-all-4.1.17.Final.jar,/root/angel-3.0.1-bin/bin/..//lib/angel-ps-mllib-3.0.1.jar,/root/angel-3.0.1-bin/bin/..//lib/angel-ps-tools-3.0.1.jar,/root/angel-3.0.1-bin/bin/..//lib/scala-reflect-2.11.8.jar,/root/angel-3.0.1-bin/bin/..//lib/memory-0.8.1.jar,/root/angel-3.0.1-bin/bin/..//lib/sketches-core-0.8.1.jar,/root/angel-3.0.1-bin/bin/..//lib/commons-pool-1.6.jar,/root/angel-3.0.1-bin/bin/..//lib/kryo-shaded-4.0.0.jar,/root/angel-3.0.1-bin/bin/..//lib/kryo-serializers-0.42.jar,/root/angel-3.0.1-bin/bin/..//lib/scala-library-2.11.8.jar,/root/angel-3.0.1-bin/bin/..//lib/angel-ps-core-3.0.1.jar,/root/angel-3.0.1-bin/bin/..//lib/angel-ps-psf-3.0.1.jar,/root/angel-3.0.1-bin/bin/..//lib/fastutil-7.1.0.jar,/root/angel-3.0.1-bin/bin/..//lib/sizeof-0.3.0.jar,/root/angel-3.0.1-bin/bin/..//lib/minlog-1.3.0.jar,/root/angel-3.0.1-bin/bin/..//lib/breeze_2.11-0.13.jar,/root/angel-3.0.1-bin/bin/..//lib/angel-format-0.1.1.jar,/root/angel-3.0.1-bin/bin/..//lib/angel-math-0.1.1.jar,/root/angel-3.0.1-bin/bin/..//lib/angel-mlcore-0.1.2.jar,/root/angel-3.0.1-bin/bin/..//lib/commons-math-2.2.jar
22/01/12 11:20:32 INFO utils.AngelRunJar: jars loaded: file:///root/angel-3.0.1-bin/bin/..//lib/angel-math-0.1.1.jar,file:///root/angel-3.0.1-bin/bin/..//lib/angel-format-0.1.1.jar,file:///root/angel-3.0.1-bin/bin/..//lib/angel-mlcore-0.1.2.jar,file:///root/angel-3.0.1-bin/bin/..//lib/jniloader-1.1.jar,file:///root/angel-3.0.1-bin/bin/..//lib/native_system-java-1.1.jar,file:///root/angel-3.0.1-bin/bin/..//lib/arpack_combined_all-0.1.jar,file:///root/angel-3.0.1-bin/bin/..//lib/all-1.1.2.pom,file:///root/angel-3.0.1-bin/bin/..//lib/core-1.1.2.jar,file:///root/angel-3.0.1-bin/bin/..//lib/netlib-native_ref-linux-armhf-1.1-natives.jar,file:///root/angel-3.0.1-bin/bin/..//lib/netlib-native_ref-linux-i686-1.1-natives.jar,file:///root/angel-3.0.1-bin/bin/..//lib/netlib-native_ref-linux-x86_64-1.1-natives.jar,file:///root/angel-3.0.1-bin/bin/..//lib/netlib-native_system-linux-armhf-1.1-natives.jar,file:///root/angel-3.0.1-bin/bin/..//lib/netlib-native_system-linux-i686-1.1-natives.jar,file:///root/angel-3.0.1-bin/bin/..//lib/netlib-native_system-linux-x86_64-1.1-natives.jar,file:///root/angel-3.0.1-bin/bin/..//lib/jackson-annotations-2.6.5.jar,file:///root/angel-3.0.1-bin/bin/..//lib/jackson-core-2.7.7.jar,file:///root/angel-3.0.1-bin/bin/..//lib/jackson-core-asl-1.9.13.jar,file:///root/angel-3.0.1-bin/bin/..//lib/jackson-databind-2.7.7.jar,file:///root/angel-3.0.1-bin/bin/..//lib/jackson-jaxrs-1.9.13.jar,file:///root/angel-3.0.1-bin/bin/..//lib/jackson-mapper-asl-1.9.13.jar,file:///root/angel-3.0.1-bin/bin/..//lib/jackson-module-paranamer-2.6.5.jar,file:///root/angel-3.0.1-bin/bin/..//lib/jackson-module-scala_2.11-2.6.5.jar,file:///root/angel-3.0.1-bin/bin/..//lib/jackson-xc-1.9.13.jar,file:///root/angel-3.0.1-bin/bin/..//lib/json4s-ast_2.11-3.2.11.jar,file:///root/angel-3.0.1-bin/bin/..//lib/json4s-core_2.11-3.2.11.jar,file:///root/angel-3.0.1-bin/bin/..//lib/json4s-jackson_2.11-3.2.11.jar,file:///root/angel-3.0.1-bin/bin/..//lib/netty-all-4.1.17.Final.jar,file:///root/angel-3.0.1-bin/bin/..//lib/angel-ps-mllib-3.0.1.jar,file:///root/angel-3.0.1-bin/bin/..//lib/angel-ps-tools-3.0.1.jar,file:///root/angel-3.0.1-bin/bin/..//lib/scala-reflect-2.11.8.jar,file:///root/angel-3.0.1-bin/bin/..//lib/memory-0.8.1.jar,file:///root/angel-3.0.1-bin/bin/..//lib/sketches-core-0.8.1.jar,file:///root/angel-3.0.1-bin/bin/..//lib/commons-pool-1.6.jar,file:///root/angel-3.0.1-bin/bin/..//lib/kryo-shaded-4.0.0.jar,file:///root/angel-3.0.1-bin/bin/..//lib/kryo-serializers-0.42.jar,file:///root/angel-3.0.1-bin/bin/..//lib/scala-library-2.11.8.jar,file:///root/angel-3.0.1-bin/bin/..//lib/angel-ps-core-3.0.1.jar,file:///root/angel-3.0.1-bin/bin/..//lib/angel-ps-psf-3.0.1.jar,file:///root/angel-3.0.1-bin/bin/..//lib/fastutil-7.1.0.jar,file:///root/angel-3.0.1-bin/bin/..//lib/sizeof-0.3.0.jar,file:///root/angel-3.0.1-bin/bin/..//lib/minlog-1.3.0.jar,file:///root/angel-3.0.1-bin/bin/..//lib/breeze_2.11-0.13.jar,file:///root/angel-3.0.1-bin/bin/..//lib/angel-format-0.1.1.jar,file:///root/angel-3.0.1-bin/bin/..//lib/angel-math-0.1.1.jar,file:///root/angel-3.0.1-bin/bin/..//lib/angel-mlcore-0.1.2.jar,file:///root/angel-3.0.1-bin/bin/..//lib/commons-math-2.2.jar
22/01/12 11:20:32 INFO utils.AngelRunJar: angel python file: null
22/01/12 11:20:32 INFO utils.UGITools: UGI_PROPERTY_NAME is null
22/01/12 11:20:32 INFO utils.AngelRunJar: submitClass: com.tencent.angel.ml.core.graphsubmit.GraphRunner
22/01/12 11:20:33 INFO utils.UGITools: UGI_PROPERTY_NAME is null
22/01/12 11:20:33 INFO yarn.AngelYarnClient: userName:root,stagingDir:,stagingDir, conf:Configuration: core-default.xml, core-site.xml, mapred-default.xml, mapred-site.xml, yarn-default.xml, yarn-site.xml, /usr/hdp/3.1.4.0-315/hadoop/etc/hadoop/yarn-site.xml, /usr/hdp/3.1.4.0-315/hadoop/etc/hadoop/hdfs-site.xml, /root/angel-3.0.1-bin/conf/angel-site.xml
22/01/12 11:20:33 INFO client.RMProxy: Connecting to ResourceManager at slaves01/172.13.1.95:8050
22/01/12 11:20:34 INFO client.AngelClient: running mode = ANGEL_PS_WORKER
22/01/12 11:20:34 INFO utils.HdfsUtil: tmp output dir is hdfs://slaves01:8020/tmp/root/application_1641874296476_0015_a84d16c7-bddf-4ab9-905e-ff2eac84d361
22/01/12 11:20:34 INFO utils.HdfsUtil: tmp output dir is hdfs://slaves01:8020/tmp/root/application_1641874296476_0015_5cef412a-67a2-4cb2-bbc4-a684f3f0b280
22/01/12 11:20:34 INFO client.AngelClient: angel.tmp.output.path=hdfs://slaves01:8020/tmp/root/application_1641874296476_0015_a84d16c7-bddf-4ab9-905e-ff2eac84d361
22/01/12 11:20:34 INFO client.AngelClient: internal state file is hdfs://slaves01:8020/tmp/root/application_1641874296476_0015_5cef412a-67a2-4cb2-bbc4-a684f3f0b280/state
22/01/12 11:20:34 INFO yarn.AngelYarnClient: default FileSystem: hdfs://slaves01:8020
22/01/12 11:20:34 INFO yarn.AngelYarnClient: libjarsDir=/tmp/hadoop-yarn/root/.staging/application_1641874296476_0015/libjars
22/01/12 11:20:34 INFO yarn.AngelYarnClient: libjars=file:///root/angel-3.0.1-bin/bin/..//lib/angel-math-0.1.1.jar,file:///root/angel-3.0.1-bin/bin/..//lib/angel-format-0.1.1.jar,file:///root/angel-3.0.1-bin/bin/..//lib/angel-mlcore-0.1.2.jar,file:///root/angel-3.0.1-bin/bin/..//lib/jniloader-1.1.jar,file:///root/angel-3.0.1-bin/bin/..//lib/native_system-java-1.1.jar,file:///root/angel-3.0.1-bin/bin/..//lib/arpack_combined_all-0.1.jar,file:///root/angel-3.0.1-bin/bin/..//lib/all-1.1.2.pom,file:///root/angel-3.0.1-bin/bin/..//lib/core-1.1.2.jar,file:///root/angel-3.0.1-bin/bin/..//lib/netlib-native_ref-linux-armhf-1.1-natives.jar,file:///root/angel-3.0.1-bin/bin/..//lib/netlib-native_ref-linux-i686-1.1-natives.jar,file:///root/angel-3.0.1-bin/bin/..//lib/netlib-native_ref-linux-x86_64-1.1-natives.jar,file:///root/angel-3.0.1-bin/bin/..//lib/netlib-native_system-linux-armhf-1.1-natives.jar,file:///root/angel-3.0.1-bin/bin/..//lib/netlib-native_system-linux-i686-1.1-natives.jar,file:///root/angel-3.0.1-bin/bin/..//lib/netlib-native_system-linux-x86_64-1.1-natives.jar,file:///root/angel-3.0.1-bin/bin/..//lib/jackson-annotations-2.6.5.jar,file:///root/angel-3.0.1-bin/bin/..//lib/jackson-core-2.7.7.jar,file:///root/angel-3.0.1-bin/bin/..//lib/jackson-core-asl-1.9.13.jar,file:///root/angel-3.0.1-bin/bin/..//lib/jackson-databind-2.7.7.jar,file:///root/angel-3.0.1-bin/bin/..//lib/jackson-jaxrs-1.9.13.jar,file:///root/angel-3.0.1-bin/bin/..//lib/jackson-mapper-asl-1.9.13.jar,file:///root/angel-3.0.1-bin/bin/..//lib/jackson-module-paranamer-2.6.5.jar,file:///root/angel-3.0.1-bin/bin/..//lib/jackson-module-scala_2.11-2.6.5.jar,file:///root/angel-3.0.1-bin/bin/..//lib/jackson-xc-1.9.13.jar,file:///root/angel-3.0.1-bin/bin/..//lib/json4s-ast_2.11-3.2.11.jar,file:///root/angel-3.0.1-bin/bin/..//lib/json4s-core_2.11-3.2.11.jar,file:///root/angel-3.0.1-bin/bin/..//lib/json4s-jackson_2.11-3.2.11.jar,file:///root/angel-3.0.1-bin/bin/..//lib/netty-all-4.1.17.Final.jar,file:///root/angel-3.0.1-bin/bin/..//lib/angel-ps-mllib-3.0.1.jar,file:///root/angel-3.0.1-bin/bin/..//lib/angel-ps-tools-3.0.1.jar,file:///root/angel-3.0.1-bin/bin/..//lib/scala-reflect-2.11.8.jar,file:///root/angel-3.0.1-bin/bin/..//lib/memory-0.8.1.jar,file:///root/angel-3.0.1-bin/bin/..//lib/sketches-core-0.8.1.jar,file:///root/angel-3.0.1-bin/bin/..//lib/commons-pool-1.6.jar,file:///root/angel-3.0.1-bin/bin/..//lib/kryo-shaded-4.0.0.jar,file:///root/angel-3.0.1-bin/bin/..//lib/kryo-serializers-0.42.jar,file:///root/angel-3.0.1-bin/bin/..//lib/scala-library-2.11.8.jar,file:///root/angel-3.0.1-bin/bin/..//lib/angel-ps-core-3.0.1.jar,file:///root/angel-3.0.1-bin/bin/..//lib/angel-ps-psf-3.0.1.jar,file:///root/angel-3.0.1-bin/bin/..//lib/fastutil-7.1.0.jar,file:///root/angel-3.0.1-bin/bin/..//lib/sizeof-0.3.0.jar,file:///root/angel-3.0.1-bin/bin/..//lib/minlog-1.3.0.jar,file:///root/angel-3.0.1-bin/bin/..//lib/breeze_2.11-0.13.jar,file:///root/angel-3.0.1-bin/bin/..//lib/angel-format-0.1.1.jar,file:///root/angel-3.0.1-bin/bin/..//lib/angel-math-0.1.1.jar,file:///root/angel-3.0.1-bin/bin/..//lib/angel-mlcore-0.1.2.jar,file:///root/angel-3.0.1-bin/bin/..//lib/commons-math-2.2.jar
22/01/12 11:20:36 INFO conf.Configuration: found resource resource-types.xml at file:/etc/hadoop/3.1.4.0-315/0/resource-types.xml
AppMaster capability =
22/01/12 11:20:36 INFO yarn.AngelYarnClient: Command to launch container for ApplicationMaster is : $JAVA_HOME/bin/java -Dlog4j.configuration=log/angel.properties -Dlog4j.logger.com.tencent.ml=DEBUG -Dyarn.app.container.log.dir= -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Xmx1536M -Xms1536M -XX:PermSize=100M -XX:MaxPermSize=200M -XX:+PrintGCDateStamps -XX:+PrintGCDetails -XX:+PrintCommandLineFlags -XX:+PrintTenuringDistribution -XX:+PrintAdaptiveSizePolicy -Xloggc:/gc.log com.tencent.angel.master.AngelApplicationMaster 1>/stdout 2>/stderr
22/01/12 11:20:36 INFO util.AngelApps: yarn.application.classpath=$HADOOP_CONF_DIR,/usr/hdp/3.1.4.0-315/hadoop/*,/usr/hdp/3.1.4.0-315/hadoop/lib/*,/usr/hdp/current/hadoop-hdfs-client/*,/usr/hdp/current/hadoop-hdfs-client/lib/*,/usr/hdp/current/hadoop-yarn-client/*,/usr/hdp/current/hadoop-yarn-client/lib/*
22/01/12 11:20:36 INFO util.AngelApps: mapreduce.application.classpath=$PWD/mr-framework/hadoop/share/hadoop/mapreduce/*:$PWD/mr-framework/hadoop/share/hadoop/mapreduce/lib/*:$PWD/mr-framework/hadoop/share/hadoop/common/*:$PWD/mr-framework/hadoop/share/hadoop/common/lib/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/*:$PWD/mr-framework/hadoop/share/hadoop/yarn/lib/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/*:$PWD/mr-framework/hadoop/share/hadoop/hdfs/lib/*:$PWD/mr-framework/hadoop/share/hadoop/tools/lib/*:/usr/hdp//hadoop/lib/hadoop-lzo-0.6.0..jar:/etc/hadoop/conf/secure
22/01/12 11:20:36 INFO yarn.AngelYarnClient: ApplicationSubmissionContext Queuename : default
22/01/12 11:20:37 INFO impl.YarnClientImpl: Submitted application application_1641874296476_0015
22/01/12 11:20:43 ERROR yarn.AngelYarnClient: submit application to yarn failed.
java.io.IOException: Failed to run job : Application application_1641874296476_0015 failed 2 times due to AM Container for appattempt_1641874296476_0015_000002 exited with exitCode: 1
Failing this attempt.Diagnostics: [2022-01-12 11:20:42.801]Exception from container-launch.
Container id: container_1641874296476_0015_02_000001
Exit code: 1