I'm trying to use moto to implement integration testing and trying to test some custom resources to be implemented with cloudformation. I'm using the example provided in the github page. I've slightly altered the example to fit my development environment and to accept yaml files instead of jsons which also fits my deployment strategy. I've gone over the documentation and the code in the example and have launched a server which is whats needed by the cf.create_stack command. The example calls a method
def wait_for_log_msg(expected_msg, log_group):
...
The method returns True and a set if the expected_msg is in the received messages, else it returns False and a set (The code is found below).
PROBLEM:
As per the example the message sent is Status code: 200, That message is not received and when I print through the messages I get failed executing http.request. I'm under the assumption that this is due to some faulty approach I have with motoserver.
QUESTION:
How can I get motoserver and specifically the logstream to output the desired Status code: 200 message?
ATTEMPTS:
I tried starting motoserver using docker but I got the following error:
{"error running docker: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))"}
I then switched to running motoserver directly by calling it in the terminal using
moto_server -p5000
which yields a failure and when I print the log message I get (after formatting to make it easier to read):
{
'\x1b[32mEND RequestId: 50fe51a1-46e2-16d2-c274-8a50604d2be9\x1b[0m',
"send(..) failed executing http.request(..): HTTPConnectionPool(host='host.docker.internal', port=5000): Max retries exceeded with url: /cloudformation_us-east-1/cfnresponse?stack=arn:aws:cloudformation:us-east-1:123456789012:stack/stack10d4d2/ce9420fc-d0e5-4e71-949b-893537701776 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f58b9eabf40>: Failed to establish a new connection: [Errno 111] Connection refused'))",
'\x1b[32mSTART RequestId: 50fe51a1-46e2-16d2-c274-8a50604d2be9 Version: $LATEST\x1b[0m',
'',
"[WARNING]\t2022-05-01T05:07:18.969Z\t50fe51a1-46e2-16d2-c274-8a50604d2be9\tRetrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f58b9eab520>: Failed to establish a new connection: [Errno 111] Connection refused')': /cloudformation_us-east-1/cfnresponse?stack=arn:aws:cloudformation:us-east-1:123456789012:stack/stack10d4d2/ce9420fc-d0e5-4e71-949b-893537701776",
"[WARNING]\t2022-05-01T05:07:18.968Z\t50fe51a1-46e2-16d2-c274-8a50604d2be9\tRetrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f58b9eab1f0>: Failed to establish a new connection: [Errno 111] Connection refused')': /cloudformation_us-east-1/cfnresponse?stack=arn:aws:cloudformation:us-east-1:123456789012:stack/stack10d4d2/ce9420fc-d0e5-4e71-949b-893537701776",
'\x1b[32mREPORT RequestId: 50fe51a1-46e2-16d2-c274-8a50604d2be9\tInit Duration: 183.50 ms\tDuration: 8.25 ms\tBilled Duration: 9 ms\tMemory Size: 128 MB\tMax Memory Used: 27 MB\t\x1b[0m',
"{
'RequestType': 'Create',
'ServiceToken': 'arn:aws:lambda:us-east-1:123456789012:function:stack10d4d2-InfoFunction-M4ZP2MX8XYLL',
'ResponseURL': 'http://host.docker.internal:5000/cloudformation_us-east-1/cfnresponse?stack=arn:aws:cloudformation:us-east-1:123456789012:stack/stack10d4d2/ce9420fc-d0e5-4e71-949b-893537701776',
'StackId': 'arn:aws:cloudformation:us-east-1:123456789012:stack/stack10d4d2/ce9420fc-d0e5-4e71-949b-893537701776',
'RequestId': '046dc61f-d786-4672-b1a9-48eb286593e2',
'LogicalResourceId': 'CustomInfo',
'ResourceType': 'Custom::Info',
'ResourceProperties': {
'ServiceToken': 'arn:aws:lambda:us-east-1:123456789012:function:stack10d4d2-InfoFunction-M4ZP2MX8XYLL',
'Region': 'us-east-1',
'MyProperty': 'stuff'
}
}",
'http://host.docker.internal:5000/cloudformation_us-east-1/cfnresponse?stack=arn:aws:cloudformation:us-east-1:123456789012:stack/stack10d4d2/ce9420fc-d0e5-4e71-949b-893537701776',
'{
"Status": "SUCCESS",
"Reason": "See the details in CloudWatch Log Stream: 2022/05/01/[$LATEST]b39c266d1989dc1bc8ec6dfecf2154d7",
"PhysicalResourceId": "2022/05/01/[$LATEST]b39c266d1989dc1bc8ec6dfecf2154d7",
"StackId": "arn:aws:cloudformation:us-east-1:123456789012:stack/stack10d4d2/ce9420fc-d0e5-4e71-949b-893537701776",
"RequestId": "046dc61f-d786-4672-b1a9-48eb286593e2",
"LogicalResourceId": "CustomInfo",
"NoEcho": false,
"Data": {
"info_value": "special value"
}
}',
'Response body:',
'null',
"[WARNING]\t2022-05-01T05:07:18.970Z\t50fe51a1-46e2-16d2-c274-8a50604d2be9\tRetrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f58b9eab730>: Failed to establish a new connection: [Errno 111] Connection refused')': /cloudformation_us-east-1/cfnresponse?stack=arn:aws:cloudformation:us-east-1:123456789012:stack/stack10d4d2/ce9420fc-d0e5-4e71-949b-893537701776"
}
Of which I think the most interesting part is:
"send(..) failed executing http.request(..): HTTPConnectionPool(host='host.docker.internal', port=5000): Max retries exceeded with url: /cloudformation_us-east-1/cfnresponse?stack=arn:aws:cloudformation:us-east-1:123456789012:stack/stack10d4d2/ce9420fc-d0e5-4e71-949b-893537701776 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f58b9eabf40>: Failed to establish a new connection: [Errno 111] Connection refused'))",
CODE:
The code for the tests
# Libraries
# Standard Libraries
import inspect
import json
import os
import sys
import time
from uuid import uuid4
import yaml
# Paths
PACKAGE_PARENT = ".."
SCRIPT_DIR = os.path.dirname(
os.path.realpath(os.path.join(os.getcwd(), os.path.expanduser(__file__)))
)
sys.path.append(os.path.normpath(os.path.join(SCRIPT_DIR, PACKAGE_PARENT)))
# Third Party Librairies
import boto3
from moto import mock_cloudformation, mock_lambda, mock_logs, mock_s3, settings
import pytest
from pytest import assume
# User Defined Libraries
from .fixtures.custom_lambda import get_template
import src.app.App as lambda_function
from .test_awslambda.utilities import wait_for_log_msg
def get_log_group_name(cf, stack_name):
...
def get_outputs(cf, stack_name):
...
@pytest.fixture(scope="function")
def aws_credentials():
"""
Mocked AWS Credentials for moto.
"""
os.environ["AWS_ACCESS_KEY_ID"] = "testing"
os.environ["AWS_SECRET_ACCESS_KEY"] = "testing"
os.environ["AWS_SECURITY_TOKEN"] = "testing"
os.environ["AWS_SESSION_TOKEN"] = "testing"
os.environ["AWS_DEFAULT_REGION"] = "us-east-1"
@mock_cloudformation
@mock_lambda
@mock_logs
@mock_s3
def test_create_custom_lambda_resource():
#########
# Integration test using a Custom Resource
# Create a Lambda
# CF will call the Lambda
# The Lambda should call CF, to indicate success (using the cfnresponse-module)
# This HTTP request will include any outputs that are now stored against the stack
# TEST: verify that this output is persisted
##########
if not settings.TEST_SERVER_MODE:
raise pytest.skip(
"Needs a standalone MotoServer, as cfnresponse needs to connect to something"
)
# Create cloudformation stack
stack_name = f"stack{str(uuid4())[0:6]}"
code = inspect.getsource(lambda_function)
template_body = get_template(code)
cf = boto3.client(
"cloudformation",
region_name="us-east-1",
endpoint_url="http://localhost:5000",
)
cf.create_stack(
StackName=stack_name,
TemplateBody=template_body,
# TemplateBody=json.dumps(template_body),
Capabilities=["CAPABILITY_IAM"],
)
# Verify CloudWatch contains the correct logs
log_group_name = get_log_group_name(cf, stack_name)
success, logs = wait_for_log_msg(
expected_msg="Status code: 200", log_group=log_group_name
)
print(f"Logs should indicate success: \n{logs}")
# Verify the correct Output was returned
outputs = get_outputs(cf, stack_name)
with assume:
assert success is True
with assume:
assert len(outputs) == 1
all of which is nearly a copy paste of what is available on the github page where I change the code to import the function from another file using inspect feed it to a yaml version of a cf template and pass it attempt to mock the creation of a custom resource
For completeness wait_for_log_msg:
def wait_for_log_msg(expected_msg, log_group):
logs_conn = boto3.client("logs", region_name="us-east-1")
received_messages = []
start = time.time()
while (time.time() - start) < 30:
try:
result = logs_conn.describe_log_streams(logGroupName=log_group)
log_streams = result.get("logStreams")
except ClientError:
log_streams = None # LogGroupName does not yet exist
if not log_streams:
time.sleep(1)
continue
for log_stream in log_streams:
result = logs_conn.get_log_events(
logGroupName=log_group,
logStreamName=log_stream["logStreamName"],
)
received_messages.extend(
[event["message"] for event in result.get("events")]
)
for line in received_messages:
if expected_msg in line:
return True, set(received_messages)
time.sleep(1)
return False, set(received_messages)
which is a copy paste of what is on the github page.
CodePudding user response：
By default, MotoServer is not reachable to outside connections.
The LambdaFunction used to create the CustomResource is executed inside a Docker-container, which counts as an 'outside' connection - that's why you're seeing the NewConnectionError.
Try starting the MotoServer like this, which opens the server up to outside connections:
moto_server -h 0.0.0.0
(Note that I'm not specifying the port, as 5000 is the default - adjust as necessary.)
The Docker-exception is not related to Moto, as far as I can tell - see this SO question for possible solutions: docker.errors.DockerException: Error while fetching server API version