Hadoop MapReduce job failing in launch

MapReduce job is failing with following error even though JAVA_HOME is set.

/bin/bash: /bin/java: No such file or directory

I am trying to setup hadoop (3.3.4) on my Mac M1. I have set JAVA_HOME in /etc/hadoop/hadoop-env.sh

export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk1.8.0_351.jdk/Contents/Home

I am able to add files to the HDFS but on running the sample MapReduce job - hadoop-mapreduce-examples-3.3.4.jar wordcount, or any other MapReduce job I am getting the following error in launch_container.sh,

[2022-12-09 20:39:39.415]Container exited with a non-zero exit code 127. Error file: prelaunch.err.
Last 4096 bytes of prelaunch.err :
Last 4096 bytes of stderr :
/bin/bash: /bin/java: No such file or directory

I suspect this the following line is failing and JAVA_HOME is coming empty.

launch_container.sh

exec /bin/bash -c "$JAVA_HOME/bin/java -Djava.io.tmpdir=$PWD/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/opt/homebrew/Cellar/hadoop/3.3.4/libexec/logs/userlogs/application_1670596645240_0001/container_1670596645240_0001_01_000001 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog  -Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster 1>/opt/homebrew/Cellar/hadoop/3.3.4/libexec/logs/userlogs/application_1670596645240_0001/container_1670596645240_0001_01_000001/stdout 2>/opt/homebrew/Cellar/hadoop/3.3.4/libexec/logs/userlogs/application_1670596645240_0001/container_1670596645240_0001_01_000001/stderr "

but JAVA_HOME is set in all these places

hadoop-env.sh 
~/.bash_profile 
~/.zprofile

I have tried reinstalling hadoop 3-4 times but everytime its getting stuck here. Most other solutions say to set JAVA_HOME in hadoop-env.sh, but its already set there.

Any help will be appreciated, Thanks.

CodePudding user response：

It worked after commenting the yarn.nodemanager.env-whitelist property in the yarn-site.xml

<property>
    <name>yarn.nodemanager.env-whitelist</name>
    <value>
JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAPRED_HOME
    </value>
</property>

CodePudding user response：

do not wrap in the value

you should write like this in yarn-site.xml

  <property>
    <name>yarn.nodemanager.env-whitelist</name>
    <value>JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_MAPRED_HOME</value>
  </property>