How I am here.

1. Running Command: opt/develop/spark-3.3.1$ sudo ./dev/make-distribution.sh -pHADOOP-3 Dhadoop.version=3.2.4 -Pyarn --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver -Pmesos -Pyarn -Pkubernetes The script (make-distribution.sh), apparently, is not reading the value of the variable i.e., $JAVA_HOME.

• Running Command: /opt/develop/spark-3.3.1$ echo $JAVA_HOME Shows expected result: /usr/lib/jvm/jdk-19.0.1
• Attempted: Well, added to the file /opt/develop/spark-3.3.1/sbin/spark-config.sh export JAVA_HOME="/usr/lib/jvm/jdk-19.0.1" export PATH=$PATH:$JAVA_HOME/bin did not seem to change the result of Running command. Logging-out and logging back in is also of no avail.
• Note-1: My JAVA_HOME is also set in /etc/profile.d/my-envvars file as: export JAVA_HOME="/usr/lib/jvm/jdk-19.0.1" export PATH=$PATH:$JAVA_HOME/bin
• Note-2: commands java --version and javac --version appears to be working okay.

The file: /opt/develop/dev/make-distribution.sh Reads: `

set -o pipefail
set -e
set -x

# Figure out where the Spark framework is installed
SPARK_HOME="$(cd "`dirname "$0"`/.."; pwd)"
DISTDIR="$SPARK_HOME/dist"

MAKE_TGZ=false
MAKE_PIP=false
MAKE_R=false
NAME=none
MVN="$SPARK_HOME/build/mvn"

function exit_with_usage {
  set  x
  echo "make-distribution.sh - tool for making binary distributions of Spark"
  echo ""
  echo "usage:"
  cl_options="[--name] [--tgz] [--pip] [--r] [--mvn <mvn-command>]"
  echo "make-distribution.sh $cl_options <maven build options>"
  echo "See Spark's \"Building Spark\" doc for correct Maven options."
  echo ""
  exit 1
}

# Parse arguments
while (( "$#" )); do
  case $1 in
    --tgz)
      MAKE_TGZ=true
      ;;
    --pip)
      MAKE_PIP=true
      ;;
    --r)
      MAKE_R=true
      ;;
    --mvn)
      MVN="$2"
      shift
      ;;
    --name)
      NAME="$2"
      shift
      ;;
    --help)
      exit_with_usage
      ;;
    --*)
      echo "Error: $1 is not supported"
      exit_with_usage
      ;;
    -*)
      break
      ;;
    *)
      echo "Error: $1 is not supported"
      exit_with_usage
      ;;
  esac
  shift
done

if [ -z "$JAVA_HOME" ]; then      # THIS SHOULD EVALUATE FALSE.
  # Fall back on JAVA_HOME from rpm, if found
  if [ $(command -v  rpm) ]; then
    RPM_JAVA_HOME="$(rpm -E %java_home 2>/dev/null)"
    if [ "$RPM_JAVA_HOME" != "%java_home" ]; then
      JAVA_HOME="$RPM_JAVA_HOME"
      echo "No JAVA_HOME set, proceeding with '$JAVA_HOME' learned from rpm"
    fi
  fi

  if [ -z "$JAVA_HOME" ]; then
    if [ `command -v java` ]; then
      # If java is in /usr/bin/java, we want /usr
      JAVA_HOME="$(dirname $(dirname $(which java)))"
    fi
  fi
fi


if [ -z "$JAVA_HOME" ]; then    # This SHOULD EVALUATE FALSE. APPARENTLY, IT DOES NOT
  echo "Error: JAVA_HOME is not set, cannot proceed."
  # I Should not be here, but I am!
  exit -1
fi

`

CodePudding user response：

Your problem lies within the fact that you're executing your script with sudo, which does not preserve environment variables by default.

As a small demonstration, consider the following:

someone@somewhere:~/my-path$ export SOMEVAR="testmyvar"
someone@somewhere:~/my-path$ echo $SOMEVAR
testmyvar
someone@somewhere:~/my-path$ sudo bash -c 'echo $SOMEVAR'

As you can see, when executing a command with sudo, you don't preserve all of your environment variables.

There is luckily quite an easy fix for that: use sudo -E (that does preserve environment variables, you can find that option in the man page of sudo)

So in our little example a bit higher:

someone@somewhere:~/my-path$ sudo -E bash -c 'echo $SOMEVAR'
testmyvar

You see that the -E option does what you want it to do!

Hope this fixes your problem :)