So here is my problem: airflow, Since executes a Python file that calls a php script (Symfony) to perform a created command. The command itself works fine. The execution of the latter from Airflow also works very well (with the visual display of Airflow: runs success).

When I cause an error in the symfony command (exit(1), throw,…), to see how Airflow reacts, it always shows success. How do I make him understand that the script didn't work?

Here is the python code:

dag = DAG(
        'name fill python',
        default_args={
            'start_date': datetime(2022, 8, 1),
        },
        max_active_runs = 1,
        description='one description',
        schedule_interval='0 7 * * 1',
        tags = ["tag1", "tag2"]
)
t1 = SimpleHttpOperator(
    http_conn_id='name fill json',
    task_id='name task',
    endpoint='url route symfony',
    method='GET',
    data={},
    headers={},
    dag=dag
)
t1

And the route for the Symfony command:

public function ImportMasse(string $csv, KernelInterface $kernel)
    {
        $application = new Application($kernel);
        $application->setAutoExit(false);

        $input = new ArrayInput([
            'command' => 'Import',
            'class_name' => $csv,
        ]);

        $output = new BufferedOutput();
        $application->run($input, $output);

        if(stristr($output->fetch(), '0 errors')){
             return new Response(true);
        }else{
             return new Response(false);
        }
    }

Thanking you for your return.

CodePudding user response：

The SimpleHttpOperator allows you to run a check against the requests response object. If the check returns false an AirflowException would be raise and the operator will fail.

Example:

def check_func(response):
    # Implement your check logic here
    if condition_is_ok:
        return True
    return False
    

SimpleHttpOperator(
    ...,
    response_check=lambda response: True if check_func(response) is True else False
)