Helm chart err : prometheus.counter is not prometheus.Gauge: missing method Dec

[dimashenkov]

What happened:
Helm chart pod gets error

panic: interface conversion: *prometheus.counter is not prometheus.Gauge: missing method Dec
goroutine 1 [running]:
github.com/aws/amazon-vpc-cni-k8s/cmd/cni-metrics-helper/metrics.producePrometheusMetrics({0x2cf3990, 0xc00060f0e0}, 0xc000f93a70, 0xc000810d80)
	/go/src/github.com/aws/amazon-vpc-cni-k8s/cmd/cni-metrics-helper/metrics/metrics.go:368 +0x88a

Environment: AWS EKS

• Kubernetes version (use kubectl version): 1.29
• CNI Version 1.19
• OS (e.g: cat /etc/os-release): Amazon Linux

[orsenthil]

@dimashenkov . Do you see this issue with 1.18.6. Want to verify if this change #3093 is causing this regression.

[dimashenkov]

@orsenthil I need to test, i tested with 1.19.0 its broken with 1.18.0 its works , v1.18.6 works fine

[dimashenkov]

@orsenthil @dshehbaj cni-metrics-helper 1.18.6 works fine , interesting that 1.19.0 also works on our other cluster which has Kubernetes version : 1.29
CNI Version 1.11.4

[dshehbaj]

Hi @dimashenkov,

Can you share the helm chart version of CNI Metrics Helper running both on the cluster where you see the error and the cluster where it runs fine?

[dshehbaj]

Hi @dimashenkov,

For both clusters where you see the error and do not see the error, can you share the versions CNI and CNI Metrics Helper that are running?

In CNI version 1.19, we updated the types of some metrics: https://github.com/aws/amazon-vpc-cni-k8s/pull/3093/files#diff-fea6b7a2003b8fe9b0f1f7c48d37848be31bdfffb86a7baaebd50bb645c031d5, and CNI Metrics Helper version 1.19 also imports the same types.

Due to the nature of this change, CNI Metrics Helper version 1.19 is incompatible with CNI versions 1.18.6 and older.

I am able to reproduce the error you reported if I pair CNI Metrics Helper version 1.19 with versions of CNI 1.18.6, or older.

[dimashenkov]

Hi @dshehbaj ,
devops cluster, cni v1.19.0-eksbuild.1 , helper 1.19.0 works
stage cluster , cni v1.16.0-eksbuild.1 , helper 1.19.0 does not work ,1.18.6 works
dev cluster , cni v1.11.4-eksbuild.1 , helper 1.19.0 works
prod cluster cni v1.10.4-eksbuild.1 , helper 1.19.0 works