Create apigateway permissions from openapi definitions

[iRoachie]

Describe your idea/feature/enhancement

Hey team, recently tried building out a rest API using AWS::Serverless::Api and openapi via DefinitionBody. It's great having my OpenAPI schemas converted to API gateway models and validation 💯 .

The only thing that's missing would be to have apigw permissions automatically created.

For example, the Events property on AWS::Serverless::Function automatically creates permissions for apigw to invoke the lambda.

The OpenAPI schema already defines the lambda handler via x-amazon-apigateway-integration, so it should be possible to create this permission in the background.

x-amazon-apigateway-integration:
        uri:
          Fn::Sub: arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${MyFunction.Arn}/invocations

Proposal

Parse the DefinitionBody of AWS::Serverless::Api and create a AWS::Lambda::Permission resource for the attached lambda function.

Things to consider:

1. This requires no changes to the SAM Spec

[CoshUS]

Thanks for the feature proposal. Transferring to SAM repo.

[md84419]

Agree. It's nice to be able to create the Lambda's automatically from the openapi spec. But annlying that you then have to create the permission manually.

[tirumerla]

Any update on this? Would be nice to have this feature.

[GavinZZ]

Hi @tirumerla, thanks for reminder about this feature request. I'll add it to the team's feature list for discussion. As a workaround, our team has introduced SAM Connector.

Here is an example of a api gateway to lambda function permission using SAM Connector

MyConnector:
    Type: AWS::Serverless::Connector
    Properties:
      Source:
        Id: MyApi
      Destination:
        Id: MyFunction
      Permissions:
      - Write

For see a full list of supported connection and permissions, please take a look at this link here

[tirumerla]

Thanks @GavinZZ, will test it let you know if we any questions.

[tirumerla]

Hey @GavinZZ , we are constantly running into "internal failure" when using connectors like this. If i comment out couple of them and deploy that seems to be working fine and uncommenting and deploying again works fine. I tried adding DependsOn as well to make sure there are no race conditions with ApiGatewayStage or anything. Wondering if you have any idea on why this could be.

ApiLambdaConnectors:
    Type: AWS::Serverless::Connector
    Properties:
      Source:
        Id: ApiGateway
      Destination:
        - Arn: !Sub '{{resolve:ssm:/${AWS::StackName}/${Stage}/<func_1>}}'
          Type: AWS::Lambda::Function
        - Arn: !Sub '{{resolve:ssm:/${AWS::StackName}/${Stage}/<func_2>}}'
          Type: AWS::Lambda::Function
        - Arn: !Sub '{{resolve:ssm:/${AWS::StackName}/${Stage}/<func_3>}}'
          Type: AWS::Lambda::Function
        - Arn: !Sub '{{resolve:ssm:/${AWS::StackName}/${Stage}/<func_4>}}'
          Type: AWS::Lambda::Function
        - Arn: !Sub '{{resolve:ssm:/${AWS::StackName}/${Stage}/<func_5>}}'
          Type: AWS::Lambda::Function
      Permissions:
        - Write

Tried checking cloudtrail no errors there.

Error

2023-04-05 00:30:50 UTC-0700 | staging | UPDATE_ROLLBACK_IN_PROGRESS | The following resource(s) failed to update: [ApiLambdaConnectorsWriteLambdaPermissionDestination7].
-- | -- | -- | --
2023-04-05 00:30:48 UTC-0700 | ApiLambdaConnectorsWriteLambdaPermissionDestination7 | UPDATE_FAILED | Internal Failure

[GavinZZ]

Hi @tirumerla thanks for reporting the issues. Would you be able to provide me with the full error messages? Also curious which region are you deploying the template to? Meanwhile I will try to create a template that's similar to yours and see if I can reproduce the issue.

[tirumerla]

Would you be able to provide me with the full error messages

Hey @GavinZZ , Thanks for your response. Unfortunately this is the only error i see in the cloudformation stack when deploying. I tried adding "DependsOn" to see if it's race condition doesn't seem to help. I tried commenting three destinations with ssm store it started to work and then uncommenting and redeploying works as well so not really sure if there is any limitations to number of destinations you can use for a single connector logical id.

For reference we have 10 param store keys that we are adding as destination to get the function information from the previous stacks and also 28 nested stack outputs of function ARN's that we are adding to the same connector. So in total we have 38 destination references to the serverless connector.

[GavinZZ]

@tirumerla When the deployment failed, if you navigate the CloudForamtion console and find the stack that you tried to deploy but in CREATE_FAIELD state. If you navigate to the events tab, do you see any additional error messages?

[GavinZZ]

Also curious if you use single destination format, would you still experience the same issue?

ApiLambdaConnectors1:
    Type: AWS::Serverless::Connector
    Properties:
      Source:
        Id: ApiGateway
      Destination:
        Arn:
          Fn::Sub: '{{resolve:ssm:/${AWS::StackName}/${Stage}/<func1>}}'
        Type: AWS::Lambda::Function

ApiLambdaConnectors2:
    Type: AWS::Serverless::Connector
    Properties:
      Source:
        Id: ApiGateway
      Destination:
        Arn:
          Fn::Sub: '{{resolve:ssm:/${AWS::StackName}/${Stage}/<func2>}}'
        Type: AWS::Lambda::Function

......

[tirumerla]

@tirumerla When the deployment failed, if you navigate the CloudForamtion console and find the stack that you tried to deploy but in CREATE_FAIELD state. If you navigate to the events tab, do you see any additional error messages?

This is how i got the "internal failure" error here :) yeah unfortunately it just gave "Internal Failure" nothing else on the connector.

Also curious if you use single destination format

I haven't tested but heard from another teammate of mine that he split 10 param stores in one connector and all the nested stack outputs in another connector, that seems to be working for him. I will give it a try and let you know. Sorry i just saw i haven't mentioned the region it's in us-east-1 I raised the support ticket for same!

[tirumerla]

Hey @GavinZZ just to update you , i created a test template by splitting connectors that seems to be work fine. I only tested once waiting for few more days to see if this complains again. So far it's looking god, do you think it's the limitation of number of connectors?

[GavinZZ]

That's great to know that there is a working workaround for this issue. Unfortunately I was not able to reproduce this issue with my custom template. I also used a multi destination connectors in my template with 80+ connections in one single connector. I was able to successfully deploy it without any issue for 10 times. I would probably need a reproducible template to deep dive into the issue.

[tirumerla]

That's great to know that there is a working workaround for this issue. Unfortunately I was not able to reproduce this issue with my custom template. I also used a multi destination connectors in my template with 80+ connections in one single connector. I was able to successfully deploy it without any issue for 10 times. I would probably need a reproducible template to deep dive into the issue.

Thanks for testing it out, yeah it's definitely weird that seeing that worked for you. May be some race conditions somewhere not really sure, i saw sometimes connectors were getting created immediately with APIGateway Deployment thought this could be the issue initially later i added DependsOn ApiGatewayStage that didn't seem to help so not really sure. Atleast for now we have workarounds. Thanks for your help.