Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Problem deploying with graceful shutdown #171

Open
tcastelli opened this issue Nov 2, 2022 · 2 comments
Open

Problem deploying with graceful shutdown #171

tcastelli opened this issue Nov 2, 2022 · 2 comments

Comments

@tcastelli
Copy link

tcastelli commented Nov 2, 2022

We are currently trying to deploy a basic app that needs to listen to SIGTERM before closing. The configuration file has this options (Meteor 2.5.8).

 {
              {
                namespace: 'aws:elasticbeanstalk:cloudwatch:logs',
                option: 'StreamLogs',
                value: 'true'
            },
            {
                namespace: 'aws:elasticbeanstalk:cloudwatch:logs',
                option: 'DeleteOnTerminate',
                value: 'true'
            },
            {
                namespace: 'aws:elasticbeanstalk:cloudwatch:logs:health',
                option: 'HealthStreamingEnabled',
                value: 'true'
            },
            {
                namespace: 'aws:elasticbeanstalk:cloudwatch:logs:health',
                option: 'DeleteOnTerminate',
                value: 'true'
            },
            {
                namespace: 'aws:elasticbeanstalk:environment',
                option: 'LoadBalancerIsShared',
                value: 'true'
            },
            {
                namespace: 'aws:elbv2:loadbalancer',
                option: 'SharedLoadBalancer',
                value: 'xxxx'
            },
            {
                namespace: 'aws:elasticbeanstalk:command',
                option: 'DeploymentPolicy',
                value: 'AllAtOnce'
            },

           {
               namespace: 'aws:autoscaling:updatepolicy:rollingupdate',
               option: 'RollingUpdateType',
               value: 'Immutable'
           },

The first thing we have noticed is that the deployment of a new version sometimes doesn't work, because it leaves the instance with Severe status and HealthCHeck never resolve (we see a socket hang up response in the logs), SIGTERM doesn't seem to be received, since our logging doesn't say anything. Checking the eb-hooks log says the .platform/hooks/prebuild/45node.sh is the last thing trying to do but no errors after that.

Weirdly, when the deployment works, we can see in the logs that the SIGTERM is received two times at the same time(printed two times), but everything seems to finish correctly.

ANy ideas of what could be happening? @zodern

@tcastelli
Copy link
Author

I got some useful info from zodern, saying that AllAtOnce type might not be supported. I since then switched to Immutable and the deployments are finish properly, but can't see any info in the logs about the SIGTERM being received. At this point I'm not sure if it's processed but not logged or not received at all. Will update later

@tcastelli
Copy link
Author

So, after doing serveral tests, it seems that console.logs or similar used on the process.on("SIGTERM", xxxx will not be stored in cloudwatch, but the rest of the code is being executed and working.

Also, not sure why but we still receive two SIGTERMS whenever the instance is replaced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant