-
Notifications
You must be signed in to change notification settings - Fork 748
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Non fatal but persistent warning: "Failed to create pod sandbox ... failed to assign an IP address to container." #3066
Comments
It seems to only be on the pods with security groups assigned via POD_ENI. They do not have this warning when there are no sec groups assigned |
digging into the cni logs, I get this. For this warning
however, that containerID doesn't exist anymore.
That is the IP that was assigned the pod. and all it's metadata. However, that containerID referenced in the logs doesn;t exist. pruned a few fields out to clean this output up
in ipamd.log
So the difference is this fails
and immediately after, this similar set of logs works
So what is causing this Thanks! |
the code shows
with the else being the error I see. That pod currently has that annotation.
So this seems like a race where something is responsible for adding json to the pod prior to security groups happening and it's not fast enough. |
This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 14 days |
#stillannoying |
Every pod that we startup gets this one Warning level event one time.
Failed to create pod sandbox: rpc error: code = Unknown desc = failed to setup network for sandbox "containerString": plugin type="aws-cni" name="aws-cni" failed (add): add cmd: failed to assign an IP address to container
Here are all the events for that pod.
To date, this has seems just like noise. But it happens whether this is a pod on a new host or existing.
Config params for aws-cni are
This is on a 1.30 EKS cluster.
CNI image is:
602401143452.dkr.ecr.us-east-1.amazonaws.com/amazon-k8s-cni:v1.18.3-eksbuild.3
Again, this seems non-fatal, but I'd love to eliminate warnings from my clusters events if possible
Thanks
The text was updated successfully, but these errors were encountered: