I am trying to create a simple MediaConnect job with Python. My pipeline is simple. S3Put triggers a Python lambda, and I am trying to create a simple job. I created a simple job using AWS Console and the json job is this -

{
  "Queue": "arn:aws:mediaconvert:ap-south-1:----:queues/Default",
  "UserMetadata": {},
  "Role": "arn:aws:iam::----:role/mediaConverterRole",
  "Settings": {
    "TimecodeConfig": {
      "Source": "ZEROBASED"
    },
    "OutputGroups": [
      {
        "Name": "File Group",
        "Outputs": [
          {
            "Preset": "System-Generic_Hd_Mp4_Av1_Aac_16x9_640x360p_24Hz_250Kbps_Qvbr_Vq6",
            "Extension": ".mp4",
            "NameModifier": "converted"
          }
        ],
        "OutputGroupSettings": {
          "Type": "FILE_GROUP_SETTINGS",
          "FileGroupSettings": {
            "Destination": "s3://----/"
          }
        }
      }
    ],
    "Inputs": [
      {
        "AudioSelectors": {
          "Audio Selector 1": {
            "DefaultSelection": "DEFAULT"
          }
        },
        "VideoSelector": {},
        "TimecodeSource": "ZEROBASED",
        "FileInput": "s3://----/videos/sample786.mp4"
      }
    ]
  },
  "AccelerationSettings": {
    "Mode": "DISABLED"
  },
  "StatusUpdateInterval": "SECONDS_60",
  "Priority": 0
}

Please note that the Role worked fine while using on AWS console. So far this is ok.

Now coming to my pipeline with s3Put -> Python Lambda -> MediaConnect, the infrastructure is written using Terraform. My iam.tf file -

# create a role
# reseource_type - resource_name
resource "aws_iam_role" "lambda_role" {
  name = "${local.resource_component}-lambda-role"
  assume_role_policy = jsonencode({
    "Version": "2012-10-17",
    "Statement": [{
      "Action": "sts:AssumeRole",
      "Principal": {
        "Service": "lambda.amazonaws.com"
        },
      "Effect": "Allow",
      "Sid": ""
      },
      {
      "Action": "sts:AssumeRole",
      "Principal": {
        "Service": "mediaconvert.amazonaws.com"
      },
      "Sid": "",
      "Effect": "Allow",
    }
    ]
  })
}

# create policy 
resource "aws_iam_policy" "policy" {
  name = "${local.resource_component}-lambda-policy"
  policy = jsonencode({
    "Version": "2012-10-17",
    "Statement": [
    {
        "Effect": "Allow",
        "Action": [
            "logs:*"
        ],
        "Resource": "arn:aws:logs:*:*:*"
    },
    {
        "Effect": "Allow",
        "Action": [
            "s3:*"
        ],
        "Resource": "arn:aws:s3:::*"
      }
    ]
  })
}

# attach policy to the role
resource "aws_iam_role_policy_attachment" "policy_attachment" {
  role       = "${aws_iam_role.lambda_role.name}"
  policy_arn = "${aws_iam_policy.policy.arn}"
}

The lambda code gets triggered by S3Put successfully. But the lambda throws error -

(AccessDeniedException) when calling the CreateJob operation: User: arn:aws:sts::---:assumed-role/vidstream-inputVideoProcessor-lambda-role/vidstream-inputVideoProcessor is not authorized to perform: iam:PassRole on resource: arn:aws:iam::---:role/mediaConverterRole

I have tried to find boto3 simple examples but nothing simpler is found online. The lambda Python Code is here -

import json
import logging
import boto3


# initialize logger
logger = logging.getLogger()
logger.setLevel(logging.INFO)

def handler(event, context):

    # get input bucket
    input_bucket_name = event['Records'][0]['s3']['bucket']['name']

    # get file/object name
    media_object = event['Records'][0]['s3']['object']['key']


    # open json mediaconvert template
    with open("job.json", "r") as jsonfile:
        job_object = json.load(jsonfile)

    # prepare data for mediaconvert job
    input_file = f's3://{input_bucket_name}/{media_object}'

    # edit job object
    job_object['Settings']['Inputs'][0]['FileInput'] = input_file

    # updated job object
    logger.info("updated job object")

    # Create MediaConvert client
    mediaconvert_client = boto3.client('mediaconvert')

    try:
        # try to create a job
        mediaconvert_client.create_job(**job_object)

    except Exception as e:
        logger.error(e)

    return {
        'statusCode': 200,
        'body': json.dumps(event)
    }

The boto3 MediaConvert documentation is provided by AWS

I am at a loss, no idea what to do. Is there any simpler example anyone can help me with? I just need to create a simple job with Lambda that works, no complexity.

Any kind of help will be highly appreciated.

CodePudding user response：

Okay I solved this issue by putting iam:PassRole to lambda policy.

{
      "Effect": "Allow",
      "Action": [
        "iam:PassRole"
        ],
      "Resource": "*"
    }

So the updated iam.tf file is -

# create a role
# reseource_type - resource_name
resource "aws_iam_role" "lambda_role" {
  name = "${local.resource_component}-lambda-role"
  assume_role_policy = jsonencode({
    "Version": "2012-10-17",
    "Statement": [{
      "Action": "sts:AssumeRole",
      "Principal": {
        "Service": "lambda.amazonaws.com"
        },
      "Effect": "Allow",
      "Sid": ""
      },
      {
      "Action": "sts:AssumeRole",
      "Principal": {
        "Service": "mediaconvert.amazonaws.com"
      },
      "Sid": "",
      "Effect": "Allow",
    }
    ]
  })
}

# create policy 
resource "aws_iam_policy" "policy" {
  name = "${local.resource_component}-lambda-policy"
  policy = jsonencode({
    "Version": "2012-10-17",
    "Statement": [
    {
        "Effect": "Allow",
        "Action": [
            "logs:*"
        ],
        "Resource": "arn:aws:logs:*:*:*"
    },
    {
        "Effect": "Allow",
        "Action": [
            "s3:*"
        ],
        "Resource": "arn:aws:s3:::*"
      },
    {
      "Effect": "Allow",
      "Action": [
        "iam:PassRole"
        ],
      "Resource": "*"
    }
  ]
})
}


# attach policy to the role
resource "aws_iam_role_policy_attachment" "policy_attachment" {
  role       = "${aws_iam_role.lambda_role.name}"
  policy_arn = "${aws_iam_policy.policy.arn}"
}

I first added this to lambda policy from aws console. After that worked I added this on my tf file. Be careful when editing something on console while the main infrastructure is written in IACs such as Terraform, this might cause drift if you forget what you have done.