Wondering if there is any way (blueprint) to trigger an airflow dag in MWAA on the merge of a pull request (preferably via github actions)? Thanks!

CodePudding user response：

You need to create a role in AWS :

1. set permission with policy airflow:CreateCliToken

   {
       "Action": "airflow:CreateCliToken",
       "Effect": "Allow",
       "Resource": "*"
   }
   

2. Add trusted relationship (with your account and repo)

   {
   "Version": "2012-10-17",
   "Statement": [
       {
           "Sid": "",
           "Effect": "Allow",
           "Principal": {
               "Federated": "arn:aws:iam::{account_id}:oidc-provider/token.actions.githubusercontent.com"
           },
           "Action": "sts:AssumeRoleWithWebIdentity",
           "Condition": {
               "StringLike": {
                   "token.actions.githubusercontent.com:sub": "repo:{repo-name}:*"
               }
           }
       }
   ]
   }
   

3. In github action you need to set AWS credential with role-to-assume and permission to job

permissions:
  id-token: write
  contents: read

  - name: Configure AWS credentials
    uses: aws-actions/configure-aws-credentials@v1
    with:
      role-to-assume: arn:aws:iam::{ account_id }:role/{role-name}
      aws-region: {region}

4. Call MWAA using the CLI

   This means that if you invoke dagrun right after merge you are risking that it will execute an older version of your code.

   I don't know what why you need such mechanism it's not very typical requirement but I'd advise you to not trying to force this idea into your deployment.

   To clarify: If under a specific deployment you can confirm that the code you deployed is parsed and register as dag in the database then there is no risk in doing what you are after. This is probably a very rare and unique case.