Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Invoke Lambda via Http API inside Lambda Test Tool #1349

Open
wants to merge 5 commits into
base: dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,239 @@
using System;
using System.IO;
using System.Linq;
using System.Net;
using System.Text;
using System.Text.Json;
using System.Threading.Tasks;
using Amazon.Lambda.Core;
using Amazon.Lambda.TestTool.Runtime;
using Amazon.Lambda.TestTool.Runtime.LambdaMocks;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc;

namespace Amazon.Lambda.TestTool.BlazorTester.Controllers
{
[Route("[controller]")]
public class InvokeApiController : ControllerBase
{
private readonly LocalLambdaOptions _lambdaOptions;
private LambdaConfigInfo _lambdaConfig;

public InvokeApiController(LocalLambdaOptions lambdaOptions)
{
_lambdaOptions = lambdaOptions;
}

[HttpPost("execute")]
public async Task<IActionResult> ExecuteFunction()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the purpose of this execute? Makes me nervous that running the first function info we find. I would rather always be specific.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was convenient due to the fact that it was not necessary to know the name of the function on the client side. We store each function in separate C# projects (one configuration file per project). So you only need to know port on which TestTool listens.
Maybe you are right, it's no clear. I need to look at our client side code and think about it

{
if (!TryGetConfigFile(out var lambdaConfig))
{
return StatusCode((int)HttpStatusCode.InternalServerError,
new InternalException("ServiceException", "Error while loading function configuration"));
}

if (lambdaConfig.FunctionInfos.Count == 0)
{
return NotFound(new InternalException("ResourceNotFoundException", "Default function not found"));
}

return Ok(await ExecuteFunctionInternal(lambdaConfig, lambdaConfig.FunctionInfos[0]));
}

[HttpPost("execute/{functionName}")]
Copy link
Member

@normj normj Mar 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we just have the 2015-03-31/functions/{functionName}/invocations path. Someday I would like this tool to have a sort of API Gateway emulator and I want to avoid any possible name collisions between the user's defined resource path to any of our resource paths.

Copy link
Author

@ggaller ggaller Mar 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See my comments

[HttpPost("2015-03-31/functions/{functionName}/invocations")]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The goal I assume is not just be able to make REST calls but to actually use an AWS SDK pointing at this instance. That is a cool feature but we will need to add some tests for this so we know we don't branch that emulation.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the ability to call through a simple HTTP API is primary. For example we don't use the AWS SDK in Unity, instead we have a self-written client to call Lambda functions. The idea was originally born when I saw the Blazor client and thought that if a function could be called from the UI, there might be a ready-made API, but it turned out not.
So I think of it as an API for LambdaTestTool in addition to the UI. Compatibility with AWS SDK is a nice bonus 😃

public async Task<object> ExecuteFunction(string functionName)
{
if (!TryGetConfigFile(out var lambdaConfig))
{
return StatusCode((int)HttpStatusCode.InternalServerError,
new InternalException("ServiceException", "Error while loading function configuration"));
}

var functionInfo = lambdaConfig.FunctionInfos.FirstOrDefault(f => f.Name == functionName);
if (functionInfo == null)
{
return NotFound(new InternalException("ResourceNotFoundException",
$"Function not found: {functionName}"));
}

return Ok(await ExecuteFunctionInternal(lambdaConfig, functionInfo));
}

private bool TryGetConfigFile(out LambdaConfigInfo configInfo)
{
configInfo = null;

if (_lambdaConfig != null)
{
configInfo = _lambdaConfig;
return true;
}

if (_lambdaOptions.LambdaConfigFiles.Count == 0)
{
Console.Error.WriteLine("LambdaConfigFiles list is empty");
return false;
}

var configPath = _lambdaOptions.LambdaConfigFiles[0];
try
{
configInfo = LambdaDefaultsConfigFileParser.LoadFromFile(configPath);
_lambdaConfig = configInfo;
}
catch (Exception e)
{
Console.Error.WriteLine("Error loading lambda config from '{0}'", configPath);
Console.Error.WriteLine(e.ToString());
}

return true;
}

private async Task<object> ExecuteFunctionInternal(LambdaConfigInfo lambdaConfig,
LambdaFunctionInfo functionInfo)
{
var requestReader = new LambdaRequestReader(Request);
var function = _lambdaOptions.LoadLambdaFuntion(lambdaConfig, functionInfo.Handler);

var request = new ExecutionRequest
{
Function = function,
AWSProfile = lambdaConfig.AWSProfile,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need some way on either a per invoke or per session users can configure the profile and region. This feature won't work for users that don't want profiles and regions in their config file. I think it is fine using the config as default the. Also we might want to use the SDK's credential and region fallback chain if neither the config or a specific profile and region are not specified. Also can we add a log message which profile and region were used?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In LambdaExecutor these config parameters are passed to env variables if they are set. If you don't fill it in the config file, you can pass these parameters directly via environment variables when you start debugging. What we actively use, for example, through launchSettings.json.

At what level do you want to see profile and region logging? I propose to show the values of the corresponding environment variables at LambdaExecutor level. Because these are the values that will already be used later in the execution of the function

AWSRegion = lambdaConfig.AWSRegion,
Payload = await requestReader.ReadPayload(),
ClientContext = requestReader.ReadClientContext()
};

var response = await _lambdaOptions.LambdaRuntime.ExecuteLambdaFunctionAsync(request);
var responseWriter = new LambdaResponseWriter(Response);

if (requestReader.ReadLogType() == "Tail")
{
responseWriter.WriteLogs(response.Logs);
}

if (!response.IsSuccess)
{
responseWriter.WriteError();
return new LambdaException(response.Error);
}

return response.Response;
}

private class LambdaRequestReader
{
private const string LogTypeHeader = "X-Amz-Log-Type";
private const string ClientContextHeader = "X-Amz-Client-Context";

private readonly HttpRequest _request;

public LambdaRequestReader(HttpRequest request)
{
_request = request;
}

public async Task<string> ReadPayload()
{
using var reader = new StreamReader(_request.Body);
return await reader.ReadToEndAsync();
}

public string ReadLogType()
{
return _request.Headers.TryGetValue(LogTypeHeader, out var value)
? value.ToString()
: string.Empty;
}

public IClientContext ReadClientContext()
{
if (!_request.Headers.TryGetValue(ClientContextHeader, out var contextString))
{
return null;
}

var clientContext = JsonSerializer.Deserialize<LocalClientContext>(
Convert.FromBase64String(contextString),
new JsonSerializerOptions
{
PropertyNameCaseInsensitive = true
});

return clientContext;
}
}

private class LambdaResponseWriter
{
private const string FunctionErrorHeader = "X-Amz-Function-Error";
private const string LogResultHeader = "X-Amz-Log-Result";

private readonly HttpResponse _response;

public LambdaResponseWriter(HttpResponse response)
{
_response = response;
}

public void WriteError()
{
_response.Headers[FunctionErrorHeader] = "Unhandled";
}

public void WriteLogs(string logs)
{
_response.Headers[LogResultHeader] = Convert.ToBase64String(Encoding.UTF8.GetBytes(logs));
}
}

private class InternalException
{
public string ErrorCode { get; }

public string ErrorMessage { get; }

public InternalException(string errorCode, string errorMessage)
{
ErrorCode = errorCode;
ErrorMessage = errorMessage;
}
}

private class LambdaException
{
public string ErrorType { get; }

public string ErrorMessage { get; }

public string[] StackTrace { get; }

public LambdaException(string error)
{
var errorLines = error.Split('\n', StringSplitOptions.RemoveEmptyEntries);
if (errorLines.Length == 0)
{
StackTrace = Array.Empty<string>();
return;
}

StackTrace = errorLines.Skip(1).Select(s => s.Trim()).ToArray();

var errorMessage = errorLines[0];
var errorTypeDelimiterPos = errorMessage.IndexOf(':');

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you re-write this as:

var errorMessageParts = errorMessage.Split(':');
if (errorMessageParts.Length > 1)
{
    ErrorType = errorMessageParts[0].Trim();
    ErrorMessage = errorMessageParts[1].Trim();
}

to improve readability?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree with you, fixed it.

if (errorTypeDelimiterPos > 0)
{
ErrorType = errorMessage.Substring(0, errorTypeDelimiterPos).Trim();
ErrorMessage = errorMessage.Substring(errorTypeDelimiterPos + 1).Trim();
}
else
{
ErrorMessage = errorMessage;
}
}
}
}
}
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
namespace Amazon.Lambda.TestTool.Runtime
using Amazon.Lambda.Core;
using Amazon.Lambda.TestTool.Runtime.LambdaMocks;

namespace Amazon.Lambda.TestTool.Runtime
{
/// <summary>
/// The information used to execute the Lambda function within the test tool
Expand All @@ -24,6 +27,11 @@ public class ExecutionRequest
/// The JSON payload that will be the input of the Lambda function.
/// </summary>
public string Payload { get; set; }

/// <summary>
/// ClientContext that pass to Lambda function as a part of ILambdaContext
/// </summary>
public IClientContext ClientContext { get; set; }

}
}
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,8 @@ public async Task<ExecutionResponse> ExecuteAsync(ExecutionRequest request)

var context = new LocalLambdaContext()
{
Logger = logger
Logger = logger,
ClientContext = request.ClientContext
};

object instance = null;
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
using System.Collections.Generic;
using Amazon.Lambda.Core;

namespace Amazon.Lambda.TestTool.Runtime.LambdaMocks
{
public class LocalClientContext: IClientContext
{
public IDictionary<string, string> Environment { get; set; }

public IClientApplication Client { get; set; }

public IDictionary<string, string> Custom { get; set; }
}
}