I want to run the start-all.sh file, as I have formatted the Namenode in HDFS. I am using it with Docker.

To access it I use:

docker exec -u root -it jupyter-stack /bin/bash

The start-all.sh file is located in the path:

/usr/local/spark-2.4.4-bin-hadoop2.7/sbin/

When I access that path and run it I get the following error:

bash: /usr/local/bin/start-all.sh: Permission denied

How can I fix it and be able to run all services?

CodePudding user response：

Your error refers to a different path than that of the script, so seems you should give the absolute path in the command.

However, running multiple services in the same container is bad practice in Docker.

You should use a different container for the namenode (and Datanode) such as ones distributed by user bde2020 - https://github.com/big-data-europe/docker-hadoop

Or you can use less Docker images by using alternative filesystems, such as MinIO, or simply letting Spark read from its local container filesystem