I have a job in AWS Batch, which calls a .sh file which in turn executes a python script. The python script fails with an error that it does not find a specific file in a directory, but, the process in AWS Batch finishes successfully, which should be the other way around. Is there anything I should check?

CodePudding user response：

Would need to see both of the scripts for a better understanding. But unless you have some specific catch exception methods in your python script to return an error to the shell script when faced with an error, your shell script will proceed to run normally even if your python script returns an error. For example:

#!/bin/bash

python3 main.py # calls a python script
                # python script returns an error
echo "Hellow World!" # The shell script does not know if your python script returned an error or not.
                # So it continues running the shell script

So the output would most likely be something like below:

FileNotFoundError: [Errno 2] No such file or directory: 'C:/Users/User/Desktop/Python-stuff
Hello World!

CodePudding user response：

Try passing along the exit status:

#!/bin/bash
python3 /path/to/script.py
exit $?

CodePudding user response：

Add set -euxo pipefail in the first line after the shebang of your bash script and it will fail if any of your statements fail.

More information here about what those set flags mean.

Actually to solve your question just set -e could be enough in some cases, but the other additional flags are also nice to have in most cases (as in the example below which does not fail just with set -e, you need set -eu).

One example script below:

#!/usr/bin/env bash
set -euxo pipefail
echo $UNDECLARED VARIABLE
echo "Script does not reach here"

If you run above script you will get this error: UNDECLARED: unbound variable. However without set -euxo pipefail script will exit with success error code (echo $? will return zero because the last statement was executed without errors).