I am trying to run a click cli through a bash with a different command groups run through a single command collection.

src/preprocessing_extract_data/scripts/main.py

import click 

@click.group()
def run_preprocessing_extract_data():
    pass


@run_preprocessing_extract_data.command()
@click.option(
    "--start_date",
    type=click.DateTime(formats=["%Y-%m-%d"]),
    required=True,
    help="Start date for the pipeline",
)
@click.option(
    "--end_date",
    type=click.DateTime(formats=["%Y-%m-%d"]),
    required=True,
    help="End date for the pipeline",
)
def main(start_date, end_date):
    ...

if __name__ == "__main__":
    main()

src/scripts/main.py

from click import CommandCollection

from src.preprocessing_extract_data.scripts.main import run_preprocessing_extract_data

if __name__ == "__main__":
    cmds = [
        run_preprocessing_extract_data,
        # a few more similar command groups
    ]
    cli = CommandCollection(sources=cmds)
    cli()

scripts/entrypoint.sh

#!/bin/sh
start_date="$1"
end_date="$2"

python src/scripts/main.py run_preprocessing_extract_data --start_date=$start_date --end_date=$end_date 

I run it using ./scripts/entrypoint.sh --start_date="2020-11-01" --end_date="2021-12-01" --today="2021-12-10" but it keeps failing and throws the following error:

Usage: main.py [OPTIONS] COMMAND [ARGS]...
Try 'main.py --help' for help.

Error: No such command 'run_preprocessing_extract_data'.

CodePudding user response：

From the docs:

The default implementation for such a merging system is the CommandCollection class. It accepts a list of other multi commands and makes the commands available on the same level.

Hence, your script now has a command main; you can check this by running your script with --help (or no arguments at all): python src/scripts/main.py --help.

Hence you can do the following:

python src/scripts/main.py main --start_date="$start_date" --end_date="$end_date"

By the way, invoking your shell script should be done without the --start_date: ./scripts/entrypoint.sh "2020-11-01" "2021-12-01".