Skip to content

Commit

Permalink
Merge remote-tracking branch 'upstream/develop' into integrate-upstre…
Browse files Browse the repository at this point in the history 
…am-changes

Integrate upstream changes. Notable conflicts:

* Makefile: simply upgrade Go version to 1.19 from upstream and use LocalStack changes
* go.mod: merge dependency updates from upstream and keep LocalStack changes
* go.sum: re-generate
* lambda/rapi/server.go: new interop import from upstream (?), otherwise not really conflicts just API updates and by-1-line-neighboring updates
  • Loading branch information
@joe4dev @dfangl
joe4dev authored and dfangl committed Oct 18, 2023
2 parents 605e17d + bf7e248 commit 62535dc
Show file tree
Hide file tree
Showing 126 changed files with 6,953 additions and 2,817 deletions.
2 changes: 1 addition & 1 deletion Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ compile-lambda-linux-all:
make ARCH=arm64 compile-lambda-linux

compile-with-docker:
docker run --rm --env GOPROXY=direct -v $(shell pwd):/LambdaRuntimeLocal -w /LambdaRuntimeLocal golang:1.18 make ARCH=${ARCH} compile-lambda-linux
docker run --rm --env GOPROXY=direct -v $(shell pwd):/LambdaRuntimeLocal -w /LambdaRuntimeLocal golang:1.19 make ARCH=${ARCH} compile-lambda-linux

compile-lambda-linux:
CGO_ENABLED=0 GOOS=linux GOARCH=${GO_ARCH_${ARCH}} go build -ldflags "${RELEASE_BUILD_LINKER_FLAGS}" -gcflags="${GC_FLAGS}" -o ${DESTINATION_${ARCH}} ./cmd/localstack
Expand Down
104 changes: 65 additions & 39 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
## AWS Lambda Runtime Interface Emulator
![GitHub release (latest by date)](https://img.shields.io/github/v/release/aws/aws-lambda-runtime-interface-emulator)
![GitHub go.mod Go version](https://img.shields.io/github/go-mod/go-version/aws/aws-lambda-runtime-interface-emulator)
![GitHub](https://img.shields.io/github/license/aws/aws-lambda-runtime-interface-emulator)

![Apache-2.0](https://img.shields.io/npm/l/aws-sam-local.svg)

The Lambda Runtime Interface Emulator is a proxy for Lambda’s Runtime and Extensions APIs, which allows customers to
locally test their Lambda function packaged as a container image. It is a lightweight web-server that converts
Expand All @@ -12,6 +14,21 @@ requests instead of the JSON events required for deployment to Lambda. This comp
Lambda’s orchestrator, or security and authentication configurations. You can get started by downloading and installing it on your local machine. When the Lambda Runtime API emulator is executed, a `/2015-03-31/functions/function/invocations` endpoint will be stood up within the container that you post data to it in order to invoke your function for testing.


## Content
* [Installing](#installing)
* [Getting started](#getting-started)
* [Test an image with RIE included in the image](#test-an-image-with-rie-included-in-the-image)
* [To test your Lambda function with the emulator](#to-test-your-lambda-function-with-the-emulator)
* [Build RIE into your base image](#build-rie-into-your-base-image)
* [To build the emulator into your image](#to-build-the-emulator-into-your-image)
* [Test an image without adding RIE to the image](#test-an-image-without-adding-rie-to-the-image)
* [To test an image without adding RIE to the image](#to-test-an-image-without-adding-rie-to-the-image)
* [How to configure](#how-to-configure)
* [Level of support](#level-of-support)
* [Security](#security)
* [License](#license)


## Installing

Instructions for installing AWS Lambda Runtime Interface Emulator for your platform
Expand All @@ -26,26 +43,26 @@ Instructions for installing AWS Lambda Runtime Interface Emulator for your platf

## Getting started

There are a few ways you use the Runtime Interface Emulator (RIE) to locally test your function depending on the base image used.
There are a few ways you use the Runtime Interface Emulator (RIE) to locally test your function depending on the base image used.


### Test an image with RIE included in the image

The AWS base images for Lambda include the runtime interface emulator. You can also follow these steps if you built the RIE into your alternative base image.
The AWS base images for Lambda include the runtime interface emulator. You can also follow these steps if you built the RIE into your alternative base image.

#### To test your Lambda function with the emulator

1. Build your image locally using the docker build command.
1. Build your image locally using the docker build command.

`docker build -t myfunction:latest .`

2. Run your container image locally using the docker run command.
2. Run your container image locally using the docker run command.

`docker run -p 9000:8080 myfunction:latest`

This command runs the image as a container and starts up an endpoint locally at `localhost:9000/2015-03-31/functions/function/invocations`.
This command runs the image as a container and starts up an endpoint locally at `localhost:9000/2015-03-31/functions/function/invocations`.

3. Post an event to the following endpoint using a curl command:
3. Post an event to the following endpoint using a curl command:

`curl -XPOST "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{}'`

Expand All @@ -59,10 +76,11 @@ You can build RIE into a base image. Download the RIE from GitHub to your local

1. Create a script and save it in your project directory. Set execution permissions for the script file.

The script checks for the presence of the `AWS_LAMBDA_RUNTIME_API` environment variable, which indicates the presence of the runtime API. If the runtime API is present, the script runs [the runtime interface client](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-images.html#runtimes-api-client). Otherwise, the script runs the runtime interface emulator.
The script checks for the presence of the `AWS_LAMBDA_RUNTIME_API` environment variable, which indicates the presence of the runtime API. If the runtime API is present, the script runs [the runtime interface client](https://docs.aws.amazon.com/lambda/latest/dg/runtimes-images.html#runtimes-api-client). Otherwise, the script runs the runtime interface emulator.

The following example shows a typical script for a Node.js function.
```
The following example shows a typical script for a Node.js function.

```sh
#!/bin/sh
if [ -z "${AWS_LAMBDA_RUNTIME_API}" ]; then
exec /usr/local/bin/aws-lambda-rie /usr/bin/npx aws-lambda-ric
Expand All @@ -71,74 +89,83 @@ The following example shows a typical script for a Node.js function.
fi
```

2. Download the [runtime interface emulator](https://github.com/aws/aws-lambda-runtime-interface-emulator/releases/latest) for your target architecture (`aws-lambda-rie` for x86\_64 or `aws-lambda-rie-arm64` for arm64) from GitHub into your project directory.
2. Download the [runtime interface emulator](https://github.com/aws/aws-lambda-runtime-interface-emulator/releases/latest) for your target architecture (`aws-lambda-rie` for x86\_64 or `aws-lambda-rie-arm64` for arm64) from GitHub into your project directory.

3. Install the emulator package and change `ENTRYPOINT` to run the new script by adding the following lines to your Dockerfile:

To use the default x86\_64 architecture
```
To use the default x86\_64 architecture

```dockerfile
ADD aws-lambda-rie /usr/local/bin/aws-lambda-rie
ENTRYPOINT [ "/entry_script.sh" ]
```

To use the arm64 architecture:
```
To use the arm64 architecture:

```dockerfile
ADD aws-lambda-rie-arm64 /usr/local/bin/aws-lambda-rie
ENTRYPOINT [ "/entry_script.sh" ]
```

4. Build your image locally using the docker build command.
```

```sh
docker build -t myfunction:latest .
```

5. Run your image locally using the docker run command.
```

```sh
docker run -p 9000:8080 myfunction:latest
```

### Test an image without adding RIE to the image

You install the runtime interface emulator to your local machine. When you run the container image, you set the entry point to be the emulator.
*To test an image without adding RIE to the image *

#### To test an image without adding RIE to the image

1. From your project directory, run the following command to download the RIE (x86-64 architecture) from GitHub and install it on your local machine.

```
```sh
mkdir -p ~/.aws-lambda-rie && curl -Lo ~/.aws-lambda-rie/aws-lambda-rie \
https://github.com/aws/aws-lambda-runtime-interface-emulator/releases/latest/download/aws-lambda-rie \
&& chmod +x ~/.aws-lambda-rie/aws-lambda-rie
```
```

To download the RIE for arm64 architecture, use the previous command with a different GitHub download url.

To download the RIE for arm64 architecture, use the previous command with a different GitHub download url.
```
https://github.com/aws/aws-lambda-runtime-interface-emulator/releases/latest/download/aws-lambda-rie-arm64 \
```

2. Run your Lambda image function using the docker run command.
```
docker run -d -v ~/.aws-lambda-rie:/aws-lambda -p 9000:8080 myfunction:latest
--entrypoint /aws-lambda/aws-lambda-rie <image entrypoint> <(optional) image command>`
2. Run your Lambda image function using the docker run command.

```sh
docker run -d -v ~/.aws-lambda-rie:/aws-lambda -p 9000:8080 myfunction:latest \
--entrypoint /aws-lambda/aws-lambda-rie <image entrypoint> <(optional) image command>
```

This runs the image as a container and starts up an endpoint locally at `localhost:9000/2015-03-31/functions/function/invocations`.
This runs the image as a container and starts up an endpoint locally at `localhost:9000/2015-03-31/functions/function/invocations`.

3. Post an event to the following endpoint using a curl command:
3. Post an event to the following endpoint using a curl command:

`curl -XPOST "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{}'`
```sh
curl -XPOST "http://localhost:9000/2015-03-31/functions/function/invocations" -d '{}'
```

This command invokes the function running in the container image and returns a response.

## How to configure
## How to configure

`aws-lambda-rie` can be configured through Environment Variables within the local running Image.
`aws-lambda-rie` can be configured through Environment Variables within the local running Image.
You can configure your credentials by setting:
* `AWS_ACCESS_KEY_ID`
* `AWS_SECRET_ACCESS_KEY`
* `AWS_SESSION_TOKEN`
* `AWS_REGION`

You can configure timeout by setting AWS_LAMBDA_FUNCTION_TIMEOUT to the number of seconds you want your function to timeout in.
You can configure timeout by setting `AWS_LAMBDA_FUNCTION_TIMEOUT` to the number of seconds you want your function to timeout in.

The rest of these Environment Variables can be set to match AWS Lambda's environment but are not required.
* `AWS_LAMBDA_FUNCTION_VERSION`
Expand All @@ -147,17 +174,16 @@ The rest of these Environment Variables can be set to match AWS Lambda's environ
## Level of support
You can use the emulator to test if your function code is compatible with the Lambda environment, executes successfully
and provides the expected output. For example, you can mock test events from different event sources. You can also use
it to test extensions and agents built into the container image against the Lambda Extensions API. This component
does *not *emulate* *the orchestration behavior of AWS Lambda. For example, Lambda has a network and security
configurations that will not be emulated by this component.
You can use the emulator to test if your function code is compatible with the Lambda environment, executes successfully
and provides the expected output. For example, you can mock test events from different event sources. You can also use
it to test extensions and agents built into the container image against the Lambda Extensions API. This component
does _not_ emulate the orchestration behavior of AWS Lambda. For example, Lambda has a network and security
configurations that will not be emulated by this component.
* You can use the emulator to test if your function code is compatible with the Lambda environment, runs successfully and provides the expected output.
* You can also use it to test extensions and agents built into the container image against the Lambda Extensions API.
* This component does _not_ emulate Lambda’s orchestration, or security and authentication configurations.
* The component does _not_ support X-ray and other Lambda integrations locally.
* This component does _not_ emulate Lambda’s orchestration, or security and authentication configurations.
* The component does _not_ support X-ray and other Lambda integrations locally.
* The component supports only Linux, for x86-64 and arm64 architectures.
## Security
Expand Down
31 changes: 25 additions & 6 deletions cmd/aws-lambda-rie/handlers.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,10 @@ import (
"strings"
"time"

"go.amzn.com/lambda/core/statejson"
"go.amzn.com/lambda/interop"
"go.amzn.com/lambda/rapidcore"
"go.amzn.com/lambda/rapidcore/env"

"github.com/google/uuid"

Expand All @@ -27,6 +29,19 @@ type Sandbox interface {
Invoke(responseWriter http.ResponseWriter, invoke *interop.Invoke) error
}

type InteropServer interface {
Init(i *interop.Init, invokeTimeoutMs int64) error
AwaitInitialized() error
FastInvoke(w http.ResponseWriter, i *interop.Invoke, direct bool) error
Reserve(id string, traceID, lambdaSegmentID string) (*rapidcore.ReserveResponse, error)
Reset(reason string, timeoutMs int64) (*statejson.ResetDescription, error)
AwaitRelease() (*statejson.InternalStateDescription, error)
Shutdown(shutdown *interop.Shutdown) *statejson.InternalStateDescription
InternalState() (*statejson.InternalStateDescription, error)
CurrentToken() *interop.Token
Restore(restore *interop.Restore) error
}

var initDone bool

func GetenvWithDefault(key string, defaultValue string) string {
Expand Down Expand Up @@ -57,7 +72,7 @@ func printEndReports(invokeId string, initDuration string, memorySize string, in
invokeId, invokeDuration, math.Ceil(invokeDuration), memorySize, memorySize)
}

func InvokeHandler(w http.ResponseWriter, r *http.Request, sandbox Sandbox) {
func InvokeHandler(w http.ResponseWriter, r *http.Request, sandbox Sandbox, bs interop.Bootstrap) {
log.Debugf("invoke: -> %s %s %v", r.Method, r.URL, r.Header)
bodyBytes, err := ioutil.ReadAll(r.Body)
if err != nil {
Expand All @@ -80,7 +95,7 @@ func InvokeHandler(w http.ResponseWriter, r *http.Request, sandbox Sandbox) {

if !initDone {

initStart, initEnd := InitHandler(sandbox, functionVersion, timeout)
initStart, initEnd := InitHandler(sandbox, functionVersion, timeout, bs)

// Calculate InitDuration
initTimeMS := math.Min(float64(initEnd.Sub(initStart).Nanoseconds()),
Expand All @@ -99,7 +114,6 @@ func InvokeHandler(w http.ResponseWriter, r *http.Request, sandbox Sandbox) {
TraceID: r.Header.Get("X-Amzn-Trace-Id"),
LambdaSegmentID: r.Header.Get("X-Amzn-Segment-Id"),
Payload: bytes.NewReader(bodyBytes),
CorrelationID: "invokeCorrelationID",
}
fmt.Println("START RequestId: " + invokePayload.ID + " Version: " + functionVersion)

Expand Down Expand Up @@ -166,7 +180,7 @@ func InvokeHandler(w http.ResponseWriter, r *http.Request, sandbox Sandbox) {
w.Write(invokeResp.Body)
}

func InitHandler(sandbox Sandbox, functionVersion string, timeout int64) (time.Time, time.Time) {
func InitHandler(sandbox Sandbox, functionVersion string, timeout int64, bs interop.Bootstrap) (time.Time, time.Time) {
additionalFunctionEnvironmentVariables := map[string]string{}

// Add default Env Vars if they were not defined. This is a required otherwise 1p Python2.7, Python3.6, and
Expand All @@ -189,15 +203,20 @@ func InitHandler(sandbox Sandbox, functionVersion string, timeout int64) (time.T
// pass to rapid
sandbox.Init(&interop.Init{
Handler: GetenvWithDefault("AWS_LAMBDA_FUNCTION_HANDLER", os.Getenv("_HANDLER")),
CorrelationID: "initCorrelationID",
AwsKey: os.Getenv("AWS_ACCESS_KEY_ID"),
AwsSecret: os.Getenv("AWS_SECRET_ACCESS_KEY"),
AwsSession: os.Getenv("AWS_SESSION_TOKEN"),
XRayDaemonAddress: "0.0.0.0:0", // TODO
FunctionName: GetenvWithDefault("AWS_LAMBDA_FUNCTION_NAME", "test_function"),
FunctionVersion: functionVersion,

RuntimeInfo: interop.RuntimeInfo{
ImageJSON: "{}",
Arn: "",
Version: ""},
CustomerEnvironmentVariables: additionalFunctionEnvironmentVariables,
SandboxType: interop.SandboxClassic,
Bootstrap: bs,
EnvironmentVariables: env.NewEnvironment(),
}, timeout*1000)
initEnd := time.Now()
return initStart, initEnd
Expand Down
6 changes: 4 additions & 2 deletions cmd/aws-lambda-rie/http.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,18 @@ import (
"net/http"

log "github.com/sirupsen/logrus"
"go.amzn.com/lambda/interop"
"go.amzn.com/lambda/rapidcore"
)

func startHTTPServer(ipport string, sandbox Sandbox) {
func startHTTPServer(ipport string, sandbox *rapidcore.SandboxBuilder, bs interop.Bootstrap) {
srv := &http.Server{
Addr: ipport,
}

// Pass a channel
http.HandleFunc("/2015-03-31/functions/function/invocations", func(w http.ResponseWriter, r *http.Request) {
InvokeHandler(w, r, sandbox)
InvokeHandler(w, r, sandbox.LambdaInvokeAPI(), bs)
})

// go routine (main thread waits)
Expand Down
Loading

0 comments on commit 62535dc

Please sign in to comment.