Skip to content

Commit

Permalink
Fix merge from master to release-1.18 - for VPC CNI 1.18.2 release (#…
Browse files Browse the repository at this point in the history
…2933)

* Mount /run/xtables.lock as FileOrCreate (#2841)

Otherwise, if the file doesn't already exist on the host at startup, it will be created as a directory. This breaks (among other things) `kube-proxy`, which leads to the AWS CNI not being able to reach the API-server, which leads to the node being stuck in `NotReady` state.

Co-authored-by: Senthil Kumaran <[email protected]>

* Bump github.com/onsi/ginkgo/v2 from 2.14.0 to 2.17.1 (#2864)

Bumps [github.com/onsi/ginkgo/v2](https://github.com/onsi/ginkgo) from 2.14.0 to 2.17.1.
- [Release notes](https://github.com/onsi/ginkgo/releases)
- [Changelog](https://github.com/onsi/ginkgo/blob/master/CHANGELOG.md)
- [Commits](onsi/ginkgo@v2.14.0...v2.17.1)

---
updated-dependencies:
- dependency-name: github.com/onsi/ginkgo/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Senthil Kumaran <[email protected]>

* Bump github.com/stretchr/testify from 1.8.4 to 1.9.0 (#2863)

Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.8.4 to 1.9.0.
- [Release notes](https://github.com/stretchr/testify/releases)
- [Commits](stretchr/testify@v1.8.4...v1.9.0)

---
updated-dependencies:
- dependency-name: github.com/stretchr/testify
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump github.com/prometheus/common from 0.48.0 to 0.52.2 (#2866)

Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.48.0 to 0.52.2.
- [Release notes](https://github.com/prometheus/common/releases)
- [Commits](prometheus/common@v0.48.0...v0.52.2)

---
updated-dependencies:
- dependency-name: github.com/prometheus/common
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump helm.sh/helm/v3 from 3.14.2 to 3.14.3 (#2862)

Bumps [helm.sh/helm/v3](https://github.com/helm/helm) from 3.14.2 to 3.14.3.
- [Release notes](https://github.com/helm/helm/releases)
- [Commits](helm/helm@v3.14.2...v3.14.3)

---
updated-dependencies:
- dependency-name: helm.sh/helm/v3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Senthil Kumaran <[email protected]>

* Bump golang.org/x/sys from 0.17.0 to 0.18.0 in /test/agent (#2859)

Bumps [golang.org/x/sys](https://github.com/golang/sys) from 0.17.0 to 0.18.0.
- [Commits](golang/sys@v0.17.0...v0.18.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sys
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Senthil Kumaran <[email protected]>

* Bump github.com/containernetworking/plugins from 1.4.0 to 1.4.1 (#2860)

Bumps [github.com/containernetworking/plugins](https://github.com/containernetworking/plugins) from 1.4.0 to 1.4.1.
- [Release notes](https://github.com/containernetworking/plugins/releases)
- [Commits](containernetworking/plugins@v1.4.0...v1.4.1)

---
updated-dependencies:
- dependency-name: github.com/containernetworking/plugins
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Senthil Kumaran <[email protected]>

* remove unused Dockerfile (#2869)

* remove unused Dockerfile

* update golang and dependencies to fix CVE

* Update Kops test for 1.30 (#2868)

Co-authored-by: Joseph Chen <[email protected]>

* Update .go-version to 1.22.2 to fix CVE reports. (#2870)

* CHANGELOG, chart, and manifest changes following VPC CNI v1.18.0 release (#2876)

Co-authored-by: Joseph Chen <[email protected]>

* Update changelogs and charts for v1.18.0 release (#2858) (#2881)

Co-authored-by: Joseph Chen <[email protected]>

* Improve "cni-metrics-helper" setup experience (#2874)

Co-authored-by: Senthil Kumaran <[email protected]>

* Add correct labels to CNI metrics chart. (#2889)

* Added information on the build troubleshooting. (#2890)

* Remove unused code in vpc cni init and vpc cni binary. (#2891)

* Bump golang.org/x/sys from 0.18.0 to 0.19.0 in /test/agent (#2898)

Bumps [golang.org/x/sys](https://github.com/golang/sys) from 0.18.0 to 0.19.0.
- [Commits](golang/sys@v0.18.0...v0.19.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sys
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Filter Managed ENI. (#2895)

If the SG reconcile loop runs before the ENI/IP reconcile loop it will modify the security groups as the ENI/IP reconcile hasn't had a chance to check the tags on the ENI yet.

Without relying on cache, when the SG reconcile is run, it will not update the ENI with the node.k8s.amazonaws.com/no_manage: true tag

* Merge release-1.18 to master after v1.18.1 release (#2914)

* Update changelogs and charts for v1.18.0 release (#2858)

Co-authored-by: Joseph Chen <[email protected]>

* Resolve merge conflicts from master to release 1.18 (#2885)

* Mount /run/xtables.lock as FileOrCreate (#2841)

Otherwise, if the file doesn't already exist on the host at startup, it will be created as a directory. This breaks (among other things) `kube-proxy`, which leads to the AWS CNI not being able to reach the API-server, which leads to the node being stuck in `NotReady` state.

Co-authored-by: Senthil Kumaran <[email protected]>

* Bump github.com/onsi/ginkgo/v2 from 2.14.0 to 2.17.1 (#2864)

Bumps [github.com/onsi/ginkgo/v2](https://github.com/onsi/ginkgo) from 2.14.0 to 2.17.1.
- [Release notes](https://github.com/onsi/ginkgo/releases)
- [Changelog](https://github.com/onsi/ginkgo/blob/master/CHANGELOG.md)
- [Commits](onsi/ginkgo@v2.14.0...v2.17.1)

---
updated-dependencies:
- dependency-name: github.com/onsi/ginkgo/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Senthil Kumaran <[email protected]>

* Bump github.com/stretchr/testify from 1.8.4 to 1.9.0 (#2863)

Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.8.4 to 1.9.0.
- [Release notes](https://github.com/stretchr/testify/releases)
- [Commits](stretchr/testify@v1.8.4...v1.9.0)

---
updated-dependencies:
- dependency-name: github.com/stretchr/testify
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump github.com/prometheus/common from 0.48.0 to 0.52.2 (#2866)

Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.48.0 to 0.52.2.
- [Release notes](https://github.com/prometheus/common/releases)
- [Commits](prometheus/common@v0.48.0...v0.52.2)

---
updated-dependencies:
- dependency-name: github.com/prometheus/common
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump helm.sh/helm/v3 from 3.14.2 to 3.14.3 (#2862)

Bumps [helm.sh/helm/v3](https://github.com/helm/helm) from 3.14.2 to 3.14.3.
- [Release notes](https://github.com/helm/helm/releases)
- [Commits](helm/helm@v3.14.2...v3.14.3)

---
updated-dependencies:
- dependency-name: helm.sh/helm/v3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Senthil Kumaran <[email protected]>

* Bump golang.org/x/sys from 0.17.0 to 0.18.0 in /test/agent (#2859)

Bumps [golang.org/x/sys](https://github.com/golang/sys) from 0.17.0 to 0.18.0.
- [Commits](golang/sys@v0.17.0...v0.18.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sys
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Senthil Kumaran <[email protected]>

* Bump github.com/containernetworking/plugins from 1.4.0 to 1.4.1 (#2860)

Bumps [github.com/containernetworking/plugins](https://github.com/containernetworking/plugins) from 1.4.0 to 1.4.1.
- [Release notes](https://github.com/containernetworking/plugins/releases)
- [Commits](containernetworking/plugins@v1.4.0...v1.4.1)

---
updated-dependencies:
- dependency-name: github.com/containernetworking/plugins
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Senthil Kumaran <[email protected]>

* remove unused Dockerfile (#2869)

* remove unused Dockerfile

* update golang and dependencies to fix CVE

* Update Kops test for 1.30 (#2868)

Co-authored-by: Joseph Chen <[email protected]>

* Update .go-version to 1.22.2 to fix CVE reports. (#2870)

* CHANGELOG, chart, and manifest changes following VPC CNI v1.18.0 release (#2876)

Co-authored-by: Joseph Chen <[email protected]>

* Update changelogs and charts for v1.18.0 release (#2858) (#2881)

Co-authored-by: Joseph Chen <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Kai Wohlfahrt <[email protected]>
Co-authored-by: Senthil Kumaran <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Sushmitha Ravikumar <[email protected]>
Co-authored-by: Joseph Chen <[email protected]>
Co-authored-by: Joseph Chen <[email protected]>

* Merge master to release-1.18 for v1.18.1 release (#2882)

* Mount /run/xtables.lock as FileOrCreate (#2841)

Otherwise, if the file doesn't already exist on the host at startup, it will be created as a directory. This breaks (among other things) `kube-proxy`, which leads to the AWS CNI not being able to reach the API-server, which leads to the node being stuck in `NotReady` state.

Co-authored-by: Senthil Kumaran <[email protected]>

* Bump github.com/onsi/ginkgo/v2 from 2.14.0 to 2.17.1 (#2864)

Bumps [github.com/onsi/ginkgo/v2](https://github.com/onsi/ginkgo) from 2.14.0 to 2.17.1.
- [Release notes](https://github.com/onsi/ginkgo/releases)
- [Changelog](https://github.com/onsi/ginkgo/blob/master/CHANGELOG.md)
- [Commits](onsi/ginkgo@v2.14.0...v2.17.1)

---
updated-dependencies:
- dependency-name: github.com/onsi/ginkgo/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Senthil Kumaran <[email protected]>

* Bump github.com/stretchr/testify from 1.8.4 to 1.9.0 (#2863)

Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.8.4 to 1.9.0.
- [Release notes](https://github.com/stretchr/testify/releases)
- [Commits](stretchr/testify@v1.8.4...v1.9.0)

---
updated-dependencies:
- dependency-name: github.com/stretchr/testify
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump github.com/prometheus/common from 0.48.0 to 0.52.2 (#2866)

Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.48.0 to 0.52.2.
- [Release notes](https://github.com/prometheus/common/releases)
- [Commits](prometheus/common@v0.48.0...v0.52.2)

---
updated-dependencies:
- dependency-name: github.com/prometheus/common
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump helm.sh/helm/v3 from 3.14.2 to 3.14.3 (#2862)

Bumps [helm.sh/helm/v3](https://github.com/helm/helm) from 3.14.2 to 3.14.3.
- [Release notes](https://github.com/helm/helm/releases)
- [Commits](helm/helm@v3.14.2...v3.14.3)

---
updated-dependencies:
- dependency-name: helm.sh/helm/v3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Senthil Kumaran <[email protected]>

* Bump golang.org/x/sys from 0.17.0 to 0.18.0 in /test/agent (#2859)

Bumps [golang.org/x/sys](https://github.com/golang/sys) from 0.17.0 to 0.18.0.
- [Commits](golang/sys@v0.17.0...v0.18.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sys
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Senthil Kumaran <[email protected]>

* Bump github.com/containernetworking/plugins from 1.4.0 to 1.4.1 (#2860)

Bumps [github.com/containernetworking/plugins](https://github.com/containernetworking/plugins) from 1.4.0 to 1.4.1.
- [Release notes](https://github.com/containernetworking/plugins/releases)
- [Commits](containernetworking/plugins@v1.4.0...v1.4.1)

---
updated-dependencies:
- dependency-name: github.com/containernetworking/plugins
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Senthil Kumaran <[email protected]>

* remove unused Dockerfile (#2869)

* remove unused Dockerfile

* update golang and dependencies to fix CVE

* Update Kops test for 1.30 (#2868)

Co-authored-by: Joseph Chen <[email protected]>

* Update .go-version to 1.22.2 to fix CVE reports. (#2870)

* CHANGELOG, chart, and manifest changes following VPC CNI v1.18.0 release (#2876)

Co-authored-by: Joseph Chen <[email protected]>

* Update changelogs and charts for v1.18.0 release (#2858) (#2881)

Co-authored-by: Joseph Chen <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Kai Wohlfahrt <[email protected]>
Co-authored-by: Senthil Kumaran <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Sushmitha Ravikumar <[email protected]>
Co-authored-by: Joseph Chen <[email protected]>

* CHANGELOG, chart, and manifest updates for v1.18.1 release (#2894)

Co-authored-by: Joseph Chen <[email protected]>

* Fix metrics readme

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Joseph Chen <[email protected]>
Co-authored-by: Jay Deokar <[email protected]>
Co-authored-by: Kai Wohlfahrt <[email protected]>
Co-authored-by: Senthil Kumaran <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Sushmitha Ravikumar <[email protected]>

* Update .go-version to fix GO-2024-2824 (#2911)

* Soak Test for CNI. (#2915)

* Soak Test for CNI.

Soak Test runs a fundamental test, like connectivity across pods
launched in both primary and secondary eni interfaces.

It launches pods, tests connectivity, tears them down, and repeats this
process for 1 hour. The run time configurable with how long we want to
run the soak test.

This test helps in discoverying race condition issues, compatiblity
issues with underlying AMI.

* Fix for make check.

* Bump github.com/aws/amazon-vpc-resource-controller-k8s (#2910)

Bumps [github.com/aws/amazon-vpc-resource-controller-k8s](https://github.com/aws/amazon-vpc-resource-controller-k8s) from 1.4.1 to 1.5.0.
- [Release notes](https://github.com/aws/amazon-vpc-resource-controller-k8s/releases)
- [Commits](aws/amazon-vpc-resource-controller-k8s@v1.4.1...v1.5.0)

---
updated-dependencies:
- dependency-name: github.com/aws/amazon-vpc-resource-controller-k8s
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update ENI Limits. (#2920)

* Skip Soak Test while running other tests. (#2922)

* Update golang to go1.22.3 (#2924)

* Bump k8s.io/api from 0.29.3 to 0.30.1 (#2918)

Bumps [k8s.io/api](https://github.com/kubernetes/api) from 0.29.3 to 0.30.1.
- [Commits](kubernetes/api@v0.29.3...v0.30.1)

---
updated-dependencies:
- dependency-name: k8s.io/api
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Switch to counter for awscni_no_available_ip_addresses (#2919)

Co-authored-by: Liptan Biswas <[email protected]>
Co-authored-by: Senthil Kumaran <[email protected]>

* Expose network policy log file location to be configured using helm (#2925)

* Expose network policy log file location to be configured using helm
chart values.

* Updated log file location name.

* Merge release branch release_1.18 (#2929)

* Update changelogs and charts for v1.18.0 release (#2858)

Co-authored-by: Joseph Chen <[email protected]>

* Resolve merge conflicts from master to release 1.18 (#2885)

* Mount /run/xtables.lock as FileOrCreate (#2841)

Otherwise, if the file doesn't already exist on the host at startup, it will be created as a directory. This breaks (among other things) `kube-proxy`, which leads to the AWS CNI not being able to reach the API-server, which leads to the node being stuck in `NotReady` state.

Co-authored-by: Senthil Kumaran <[email protected]>

* Bump github.com/onsi/ginkgo/v2 from 2.14.0 to 2.17.1 (#2864)

Bumps [github.com/onsi/ginkgo/v2](https://github.com/onsi/ginkgo) from 2.14.0 to 2.17.1.
- [Release notes](https://github.com/onsi/ginkgo/releases)
- [Changelog](https://github.com/onsi/ginkgo/blob/master/CHANGELOG.md)
- [Commits](onsi/ginkgo@v2.14.0...v2.17.1)

---
updated-dependencies:
- dependency-name: github.com/onsi/ginkgo/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Senthil Kumaran <[email protected]>

* Bump github.com/stretchr/testify from 1.8.4 to 1.9.0 (#2863)

Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.8.4 to 1.9.0.
- [Release notes](https://github.com/stretchr/testify/releases)
- [Commits](stretchr/testify@v1.8.4...v1.9.0)

---
updated-dependencies:
- dependency-name: github.com/stretchr/testify
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump github.com/prometheus/common from 0.48.0 to 0.52.2 (#2866)

Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.48.0 to 0.52.2.
- [Release notes](https://github.com/prometheus/common/releases)
- [Commits](prometheus/common@v0.48.0...v0.52.2)

---
updated-dependencies:
- dependency-name: github.com/prometheus/common
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump helm.sh/helm/v3 from 3.14.2 to 3.14.3 (#2862)

Bumps [helm.sh/helm/v3](https://github.com/helm/helm) from 3.14.2 to 3.14.3.
- [Release notes](https://github.com/helm/helm/releases)
- [Commits](helm/helm@v3.14.2...v3.14.3)

---
updated-dependencies:
- dependency-name: helm.sh/helm/v3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Senthil Kumaran <[email protected]>

* Bump golang.org/x/sys from 0.17.0 to 0.18.0 in /test/agent (#2859)

Bumps [golang.org/x/sys](https://github.com/golang/sys) from 0.17.0 to 0.18.0.
- [Commits](golang/sys@v0.17.0...v0.18.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sys
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Senthil Kumaran <[email protected]>

* Bump github.com/containernetworking/plugins from 1.4.0 to 1.4.1 (#2860)

Bumps [github.com/containernetworking/plugins](https://github.com/containernetworking/plugins) from 1.4.0 to 1.4.1.
- [Release notes](https://github.com/containernetworking/plugins/releases)
- [Commits](containernetworking/plugins@v1.4.0...v1.4.1)

---
updated-dependencies:
- dependency-name: github.com/containernetworking/plugins
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Senthil Kumaran <[email protected]>

* remove unused Dockerfile (#2869)

* remove unused Dockerfile

* update golang and dependencies to fix CVE

* Update Kops test for 1.30 (#2868)

Co-authored-by: Joseph Chen <[email protected]>

* Update .go-version to 1.22.2 to fix CVE reports. (#2870)

* CHANGELOG, chart, and manifest changes following VPC CNI v1.18.0 release (#2876)

Co-authored-by: Joseph Chen <[email protected]>

* Update changelogs and charts for v1.18.0 release (#2858) (#2881)

Co-authored-by: Joseph Chen <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Kai Wohlfahrt <[email protected]>
Co-authored-by: Senthil Kumaran <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Sushmitha Ravikumar <[email protected]>
Co-authored-by: Joseph Chen <[email protected]>
Co-authored-by: Joseph Chen <[email protected]>

* Merge master to release-1.18 for v1.18.1 release (#2882)

* Mount /run/xtables.lock as FileOrCreate (#2841)

Otherwise, if the file doesn't already exist on the host at startup, it will be created as a directory. This breaks (among other things) `kube-proxy`, which leads to the AWS CNI not being able to reach the API-server, which leads to the node being stuck in `NotReady` state.

Co-authored-by: Senthil Kumaran <[email protected]>

* Bump github.com/onsi/ginkgo/v2 from 2.14.0 to 2.17.1 (#2864)

Bumps [github.com/onsi/ginkgo/v2](https://github.com/onsi/ginkgo) from 2.14.0 to 2.17.1.
- [Release notes](https://github.com/onsi/ginkgo/releases)
- [Changelog](https://github.com/onsi/ginkgo/blob/master/CHANGELOG.md)
- [Commits](onsi/ginkgo@v2.14.0...v2.17.1)

---
updated-dependencies:
- dependency-name: github.com/onsi/ginkgo/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Senthil Kumaran <[email protected]>

* Bump github.com/stretchr/testify from 1.8.4 to 1.9.0 (#2863)

Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.8.4 to 1.9.0.
- [Release notes](https://github.com/stretchr/testify/releases)
- [Commits](stretchr/testify@v1.8.4...v1.9.0)

---
updated-dependencies:
- dependency-name: github.com/stretchr/testify
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump github.com/prometheus/common from 0.48.0 to 0.52.2 (#2866)

Bumps [github.com/prometheus/common](https://github.com/prometheus/common) from 0.48.0 to 0.52.2.
- [Release notes](https://github.com/prometheus/common/releases)
- [Commits](prometheus/common@v0.48.0...v0.52.2)

---
updated-dependencies:
- dependency-name: github.com/prometheus/common
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump helm.sh/helm/v3 from 3.14.2 to 3.14.3 (#2862)

Bumps [helm.sh/helm/v3](https://github.com/helm/helm) from 3.14.2 to 3.14.3.
- [Release notes](https://github.com/helm/helm/releases)
- [Commits](helm/helm@v3.14.2...v3.14.3)

---
updated-dependencies:
- dependency-name: helm.sh/helm/v3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Senthil Kumaran <[email protected]>

* Bump golang.org/x/sys from 0.17.0 to 0.18.0 in /test/agent (#2859)

Bumps [golang.org/x/sys](https://github.com/golang/sys) from 0.17.0 to 0.18.0.
- [Commits](golang/sys@v0.17.0...v0.18.0)

---
updated-dependencies:
- dependency-name: golang.org/x/sys
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Senthil Kumaran <[email protected]>

* Bump github.com/containernetworking/plugins from 1.4.0 to 1.4.1 (#2860)

Bumps [github.com/containernetworking/plugins](https://github.com/containernetworking/plugins) from 1.4.0 to 1.4.1.
- [Release notes](https://github.com/containernetworking/plugins/releases)
- [Commits](containernetworking/plugins@v1.4.0...v1.4.1)

---
updated-dependencies:
- dependency-name: github.com/containernetworking/plugins
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Senthil Kumaran <[email protected]>

* remove unused Dockerfile (#2869)

* remove unused Dockerfile

* update golang and dependencies to fix CVE

* Update Kops test for 1.30 (#2868)

Co-authored-by: Joseph Chen <[email protected]>

* Update .go-version to 1.22.2 to fix CVE reports. (#2870)

* CHANGELOG, chart, and manifest changes following VPC CNI v1.18.0 release (#2876)

Co-authored-by: Joseph Chen <[email protected]>

* Update changelogs and charts for v1.18.0 release (#2858) (#2881)

Co-authored-by: Joseph Chen <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Kai Wohlfahrt <[email protected]>
Co-authored-by: Senthil Kumaran <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Sushmitha Ravikumar <[email protected]>
Co-authored-by: Joseph Chen <[email protected]>

* CHANGELOG, chart, and manifest updates for v1.18.1 release (#2894)

Co-authored-by: Joseph Chen <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Joseph Chen <[email protected]>
Co-authored-by: Joseph Chen <[email protected]>
Co-authored-by: Jay Deokar <[email protected]>
Co-authored-by: Kai Wohlfahrt <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Sushmitha Ravikumar <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Kai Wohlfahrt <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Sushmitha Ravikumar <[email protected]>
Co-authored-by: Joseph Chen <[email protected]>
Co-authored-by: Joseph Chen <[email protected]>
Co-authored-by: guessi <[email protected]>
Co-authored-by: Jay Deokar <[email protected]>
Co-authored-by: Liptan Biswas <[email protected]>
Co-authored-by: Liptan Biswas <[email protected]>
  • Loading branch information
10 people authored May 28, 2024
1 parent cfe40bb commit 6b3beef
Show file tree
Hide file tree
Showing 31 changed files with 767 additions and 227 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/integration-tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: "1.21"
go-version: "1.22"
- name: Set up tools
run: |
# Install ginkgo version from go.mod
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/nightly-cron-tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: "1.21"
go-version: "1.22"
- name: Set up tools
run: |
# Install ginkgo version from go.mod
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/pr-automated-tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: "1.21"
go-version: "1.22"
- name: Set up tools
run: |
go install golang.org/x/lint/golint@latest
Expand Down Expand Up @@ -50,7 +50,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: "1.21"
go-version: "1.22"
- name: Build CNI images
run: make multi-arch-cni-build
- name: Build CNI Init images
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/pr-manual-tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: "1.21"
go-version: "1.22"
- name: Set up tools
run: |
# Install ginkgo version from go.mod
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: "1.21"
go-version: "1.22"
- name: Generate CNI YAML
run: make generate-cni-yaml
- name: Create eks-charts PR
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/weekly-cron-tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v3
with:
go-version: "1.21"
go-version: "1.22"
- name: Set up tools
run: |
# Install ginkgo version from go.mod
Expand Down
2 changes: 1 addition & 1 deletion .go-version
Original file line number Diff line number Diff line change
@@ -1 +1 @@
1.22.2
1.22.3
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ See [here](./docs/iam-policy.md) for required IAM policies.
* `unit-test`, `format`,`lint` and `vet` provide ways to run the respective tests/tools and should be run before submitting a PR.
* `make docker` will create a docker container using `docker buildx` that contains the finished binaries, with a tag of `amazon/amazon-k8s-cni:latest`
* `make docker-unit-tests` uses a docker container to run all unit tests.
* builds for all build and test actions run in docker containers based on `golang:1.21.5-6-gcc-al2` unless a different `GOLANG_IMAGE` tag is passed in.
* Builds for all build and test actions run in docker containers based on `.go-version` unless a different `GOLANG_IMAGE` tag is passed in.

## Components

Expand Down
1 change: 1 addition & 0 deletions charts/aws-vpc-cni/templates/daemonset.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -134,6 +134,7 @@ spec:
- --enable-network-policy={{ .Values.enableNetworkPolicy }}
- --enable-cloudwatch-logs={{ .Values.nodeAgent.enableCloudWatchLogs }}
- --enable-policy-event-logs={{ .Values.nodeAgent.enablePolicyEventLogs }}
- --log-file={{ .Values.nodeAgent.networkPolicyAgentLogFileLocation }}
- --metrics-bind-addr={{ include "aws-vpc-cni.nodeAgentMetricsBindAddr" . }}
- --health-probe-bind-addr={{ include "aws-vpc-cni.nodeAgentHealthProbeBindAddr" . }}
- --conntrack-cache-cleanup-period={{ .Values.nodeAgent.conntrackCacheCleanupPeriod }}
Expand Down
1 change: 1 addition & 0 deletions charts/aws-vpc-cni/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ nodeAgent:
privileged: true
enableCloudWatchLogs: "false"
enablePolicyEventLogs: "false"
networkPolicyAgentLogFileLocation: "/var/log/aws-routed-eni/network-policy-agent.log"
enableIpv6: "false"
metricsBindAddr: "8162"
healthProbeBindAddr: "8163"
Expand Down
64 changes: 42 additions & 22 deletions charts/cni-metrics-helper/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,22 @@ This chart provides a Kubernetes deployment for the Amazon VPC CNI Metrics Helpe
First add the EKS repository to Helm:

```shell
helm repo add eks https://aws.github.io/eks-charts
$ helm repo add eks https://aws.github.io/eks-charts
```

To install the chart with the release name `cni-metrics-helper` and default configuration:
Ensure helm repository up to date

```shell
$ helm repo update eks
```

To identify the version you are going to apply

```shell
$ helm search repo eks/cni-metrics-helper --versions
```

To install the latest chart with the release name `cni-metrics-helper` and default configuration:

```shell
$ helm install cni-metrics-helper --namespace kube-system eks/cni-metrics-helper
Expand Down Expand Up @@ -43,26 +55,34 @@ $ helm uninstall cni-metrics-helper --namespace kube-system

The following table lists the configurable parameters for this chart and their default values.

| Parameter | Description | Default |
|------------------------------|---------------------------------------------------------------|--------------------|
| fullnameOverride | Override the fullname of the chart | cni-metrics-helper |
| image.region | ECR repository region to use. Should match your cluster | us-west-2 |
| image.tag | Image tag | v1.18.1 |
| image.account | ECR repository account number | 602401143452 |
| image.domain | ECR repository domain | amazonaws.com |
| env.USE_CLOUDWATCH | Whether to export CNI metrics to CloudWatch | true |
| env.USE_PROMETHEUS | Whether to export CNI metrics to Prometheus | false |
| env.AWS_CLUSTER_ID | ID of the cluster to use when exporting metrics to CloudWatch | default |
| env.AWS_VPC_K8S_CNI_LOGLEVEL | Log verbosity level (ie. FATAL, ERROR, WARN, INFO, DEBUG) | INFO |
| env.METRIC_UPDATE_INTERVAL | Interval at which to update CloudWatch metrics, in seconds. | |
| | Metrics are published to CloudWatch at 2x the interval | 30 |
| serviceAccount.name | The name of the ServiceAccount to use | nil |
| serviceAccount.create | Specifies whether a ServiceAccount should be created | true |
| serviceAccount.annotations | Specifies the annotations for ServiceAccount | {} |
| podAnnotations | Specifies the annotations for pods | {} |
| revisionHistoryLimit | The number of revisions to keep | 10 |
| podSecurityContext | SecurityContext to set on the pod | {} |
| containerSecurityContext | SecurityContext to set on the container | {} |

| Parameter | Description | Default |
| -------------------------------|---------------------------------------------------------------|-------------------------------------|
| `affinity` | Map of node/pod affinities | `{}` |
| `fullnameOverride` | Override the fullname of the chart | `cni-metrics-helper` |
| `image.tag` | Image tag | `v1.18.1` |
| `image.domain` | ECR repository domain | `amazonaws.com` |
| `image.region` | ECR repository region to use. Should match your cluster | `us-west-2` |
| `image.account` | ECR repository account number | `602401143452` |
| `env.USE_CLOUDWATCH` | Whether to export CNI metrics to CloudWatch | `true` |
| `env.USE_PROMETHEUS` | Whether to export CNI metrics to Prometheus | `false` |
| `env.AWS_CLUSTER_ID` | ID of the cluster to use when exporting metrics to CloudWatch | `default` |
| `env.AWS_VPC_K8S_CNI_LOGLEVEL` | Log verbosity level (ie. FATAL, ERROR, WARN, INFO, DEBUG) | `INFO` |
| `env.METRIC_UPDATE_INTERVAL` | Interval at which to update CloudWatch metrics, in seconds. | |
| | Metrics are published to CloudWatch at 2x the interval | `30` |
| `serviceAccount.name` | The name of the ServiceAccount to use | `nil` |
| `serviceAccount.create` | Specifies whether a ServiceAccount should be created | `true` |
| `serviceAccount.annotations` | Specifies the annotations for ServiceAccount | `{}` |
| `podAnnotations` | Specifies the annotations for pods | `{}` |
| `revisionHistoryLimit` | The number of revisions to keep | `10` |
| `podSecurityContext` | SecurityContext to set on the pod | `{}` |
| `containerSecurityContext` | SecurityContext to set on the container | `{}` |
| `tolerations` | Optional deployment tolerations | `[]` |
| `updateStrategy` | Optional update strategy | `{}` |
| `imagePullSecrets` | Docker registry pull secret | `[]` |
| `nodeSelector` | Node labels for pod assignment | `{}` |
| `tolerations` | Optional deployment tolerations | `[]` |


Specify each parameter using the `--set key=value[,key=value]` argument to `helm install` or provide a YAML file containing the values for the above parameters:

Expand Down
2 changes: 2 additions & 0 deletions charts/cni-metrics-helper/templates/clusterrole.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: {{ include "cni-metrics-helper.fullname" . }}
labels:
{{ include "cni-metrics-helper.labels" . | indent 4 }}
rules:
- apiGroups: [""]
resources:
Expand Down
26 changes: 24 additions & 2 deletions charts/cni-metrics-helper/templates/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,11 @@ metadata:
namespace: {{ .Release.Namespace }}
labels:
k8s-app: cni-metrics-helper
{{ include "cni-metrics-helper.labels" . | indent 4 }}
spec:
{{- if .Values.updateStrategy }}
strategy: {{ toYaml .Values.updateStrategy | nindent 4 }}
{{- end }}
revisionHistoryLimit: {{ .Values.revisionHistoryLimit }}
selector:
matchLabels:
Expand All @@ -19,6 +23,8 @@ spec:
{{- end }}
{{- end }}
labels:
app.kubernetes.io/name: {{ include "cni-metrics-helper.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
k8s-app: cni-metrics-helper
spec:
containers:
Expand All @@ -36,6 +42,22 @@ spec:
name: cni-metrics-helper
image: "{{- if .Values.image.override }}{{- .Values.image.override }}{{- else }}{{- .Values.image.account }}.dkr.ecr.{{- .Values.image.region }}.{{- .Values.image.domain }}/cni-metrics-helper:{{- .Values.image.tag }}{{- end}}"
serviceAccountName: {{ template "cni-metrics-helper.serviceAccountName" . }}
{{- if .Values.podSecurityContext }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.podSecurityContext }}
securityContext: {{ toYaml .Values.podSecurityContext | nindent 8 }}
{{- end }}
{{- end }}
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
31 changes: 31 additions & 0 deletions charts/cni-metrics-helper/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -36,3 +36,34 @@ podSecurityContext: {}
containerSecurityContext: {}

podAnnotations: {}

imagePullSecrets: []

updateStrategy: {}
# type: RollingUpdate
# rollingUpdate:
# maxUnavailable: "10%"

nodeSelector: {}

tolerations: []
# - operator: Exists

affinity: {}
# nodeAffinity:
# requiredDuringSchedulingIgnoredDuringExecution:
# nodeSelectorTerms:
# - matchExpressions:
# - key: "kubernetes.io/os"
# operator: In
# values:
# - linux
# - key: "kubernetes.io/arch"
# operator: In
# values:
# - amd64
# - arm64
# - key: "eks.amazonaws.com/compute-type"
# operator: NotIn
# values:
# - fargate
11 changes: 0 additions & 11 deletions cmd/aws-vpc-cni-init/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,6 @@ import (

const (
defaultHostCNIBinPath = "/host/opt/cni/bin"
vpcCniInitDonePath = "/vpc-cni-init/done"
metadataLocalIP = "local-ipv4"
metadataMAC = "mac"
defaultDisableIPv4TcpEarlyDemux = false
Expand Down Expand Up @@ -181,17 +180,7 @@ func _main() int {
return 1
}

// TODO: In order to speed up pod launch time, VPC CNI init container is not a Kubernetes init container.
// The VPC CNI container blocks on the existence of vpcCniInitDonePath
//err = cp.TouchFile(vpcCniInitDonePath)
//if err != nil {
// log.WithError(err).Errorf("Failed to set VPC CNI init done")
// return 1
//}

log.Infof("CNI init container done")

// TODO: Since VPC CNI init container is a real container, it never exits
// time.Sleep(time.Duration(1<<63 - 1))
return 0
}
29 changes: 0 additions & 29 deletions cmd/aws-vpc-cni/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,7 @@ import (
"os/exec"
"strconv"
"strings"
"time"

"github.com/pkg/errors"
log "github.com/sirupsen/logrus"

"github.com/containernetworking/cni/pkg/types"
Expand Down Expand Up @@ -188,27 +186,6 @@ func waitForIPAM() bool {
}
}

// Wait for vpcCniInitDonePath to exist (maximum wait time is 60 seconds)
func waitForInit() error {
start := time.Now()
maxEnd := start.Add(time.Minute)
for {
// Check for existence of vpcCniInitDonePath
if _, err := os.Stat(vpcCniInitDonePath); err == nil {
// Delete the done file in case of a reboot of the node or restart of the container (force init container to run again)
if err := os.Remove(vpcCniInitDonePath); err == nil {
return nil
}
// If file deletion fails, log and allow retry
log.Errorf("Failed to delete file: %s", vpcCniInitDonePath)
}
if time.Now().After(maxEnd) {
return errors.Errorf("time exceeded")
}
time.Sleep(1 * time.Second)
}
}

func getPrimaryIP(ipv4 bool) (string, error) {
var hostIP string
var err error
Expand Down Expand Up @@ -471,12 +448,6 @@ func _main() int {
return 1
}

// Wait for init container to complete
//if err := waitForInit(); err != nil {
// log.WithError(err).Errorf("Init container failed to complete")
// return 1
//}

log.Infof("Copying config file... ")
err = generateJSON(defaultAWSconflistFile, tmpAWSconflistFile, getPrimaryIP)
if err != nil {
Expand Down
8 changes: 8 additions & 0 deletions docs/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -253,3 +253,11 @@ The [CNI image](../scripts/dockerfiles/Dockerfile.release) built for the `aws-no

See the [cni-metrics-helper README](../cmd/cni-metrics-helper/README.md).


## Build Troubleshooting

If you encouter build issues while building vpc cni, ensure you are logged into a docker registry.
For e.g.

aws ecr-public get-login-password --region us-east-1 | docker login --username AWS --password-stdin public.ecr.aws
~
Loading

0 comments on commit 6b3beef

Please sign in to comment.