Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add etcd module #2788

Merged
merged 50 commits into from
Sep 30, 2024
Merged

feat: add etcd module #2788

merged 50 commits into from
Sep 30, 2024

Conversation

mdelapenya
Copy link
Member

@mdelapenya mdelapenya commented Sep 19, 2024

What does this PR do?

This PR adds a Go module for etcd, including clustering support. This module includes options to configure:

  • nodes in the cluster
  • cluster token
  • auto TLS for cluster-node communication I removed it because it was not critical and causing issues in tests

I followed https://etcd.io/docs/v3.5/op-guide/clustering/ to accomplish it.

The module exposes methods to talk to the the client and the peer endpoints.

Why is it important?

A new module for etcd, which could unblock adoption in other technologies (gofiber's etcd storage)

@mdelapenya mdelapenya requested a review from a team as a code owner September 19, 2024 17:26
@mdelapenya mdelapenya added the feature New functionality or new behaviors on the existing one label Sep 19, 2024
@mdelapenya mdelapenya self-assigned this Sep 19, 2024
Copy link

netlify bot commented Sep 19, 2024

Deploy Preview for testcontainers-go ready!

Name Link
🔨 Latest commit bf67013
🔍 Latest deploy log https://app.netlify.com/sites/testcontainers-go/deploys/66fabb5dd6c9ec0008d68195
😎 Deploy Preview https://deploy-preview-2788--testcontainers-go.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@eddumelendez eddumelendez changed the title feat: add etc module feat: add etcd module Sep 19, 2024
modules/etcd/cmd_test.go Outdated Show resolved Hide resolved
modules/etcd/etcd.go Outdated Show resolved Hide resolved
modules/etcd/options.go Outdated Show resolved Hide resolved
modules/etcd/options.go Outdated Show resolved Hide resolved
modules/etcd/etcd.go Outdated Show resolved Hide resolved
modules/etcd/etcd.go Outdated Show resolved Hide resolved
modules/etcd/etcd.go Outdated Show resolved Hide resolved
modules/etcd/options.go Show resolved Hide resolved
modules/etcd/etcd.go Outdated Show resolved Hide resolved
modules/etcd/etcd.go Outdated Show resolved Hide resolved
modules/etcd/etcd.go Outdated Show resolved Hide resolved
modules/etcd/options.go Show resolved Hide resolved
modules/etcd/etcd.go Outdated Show resolved Hide resolved
modules/etcd/etcd.go Outdated Show resolved Hide resolved
modules/etcd/etcd.go Outdated Show resolved Hide resolved
func configureCMD(settings options) []string {
cmds := []string{"etcd"}

if len(settings.nodeNames) < 1 {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question: do we want to allow a cluster of 1 node, doesn't really make sense?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it could be the case you want a single etcd instance

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Single instance yes but not single node cluster?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know the semantical difference between being a one-node cluster Vs being one instance (e.g. one elasticsearch instance is the same as one-node elasticsearch cluster).

In our case, the code just detects there is one node/instance, and it returns a single container. When the user requests a cluster (more than one nodes), then we return a parent container with children. So I do not see the difference here.

I could picture the Run function returning just one container and an eventual RunCluster one that separates concerns. But this is the purpose of the WithNodes option, right?

Copy link
Collaborator

@stevenh stevenh Sep 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yer I don't see the need for a RunCluster, but what I saw was all the additional command line options complexity for cluster setup which could be avoided for a single node. I believe the only reason etcd supports a single node cluster is so you can add nodes at a later date, which doesn't seem like a scenario you would need for testing.

Copy link
Member Author

@mdelapenya mdelapenya Sep 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the only reason etcd supports a single node cluster is so you can add nodes at a later date

Indeed. When I started the feature it was to support two scenarios:

  • one etcd node
  • multiple etcd nodes in a cluster

No more. Wdyt if we start like this, and check with the community how it's used. For us, deprecating the option would be enough for an eventual removal of the cluster support.

Copy link
Collaborator

@stevenh stevenh Sep 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You raise a good point, do we even need cluster support, should we just keep a patch about and add it later if there is demand, that avoids the hassle of deprecation?

It would also removed the need for ClientEndpoints and PeerEndpoints too by the looks

Copy link
Member Author

@mdelapenya mdelapenya Sep 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me add a removal commit that allows us to go back to it later.

I recalled that I ported the Java implementation. Should we still start simple?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think so, I might be wrong. I guess someone might want to test their app worked correctly if they killed a cluster node, but that would be an edge case and need additional work to expose the nodes themselves.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@eddumelendez @kiview what are your thoughts about feature parity across modules in the libraries? Should we start simple with no cluster support, or just keep what this PR does?

modules/etcd/options.go Outdated Show resolved Hide resolved
Copy link
Collaborator

@stevenh stevenh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Making some good progress, nice catch on the network clean up.

modules/etcd/etcd.go Outdated Show resolved Hide resolved
modules/etcd/etcd.go Show resolved Hide resolved
modules/etcd/etcd.go Outdated Show resolved Hide resolved
modules/etcd/etcd.go Outdated Show resolved Hide resolved
modules/etcd/etcd.go Outdated Show resolved Hide resolved
modules/etcd/etcd.go Outdated Show resolved Hide resolved
modules/etcd/etcd_unit_test.go Outdated Show resolved Hide resolved
modules/etcd/etcd_unit_test.go Show resolved Hide resolved
Copy link
Collaborator

@stevenh stevenh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few little things then I think we're there.

modules/etcd/etcd.go Outdated Show resolved Hide resolved
modules/etcd/etcd.go Outdated Show resolved Hide resolved
modules/etcd/etcd.go Outdated Show resolved Hide resolved
modules/etcd/etcd.go Outdated Show resolved Hide resolved
modules/etcd/etcd.go Show resolved Hide resolved
modules/etcd/etcd.go Show resolved Hide resolved
stevenh
stevenh previously approved these changes Sep 30, 2024
Copy link
Collaborator

@stevenh stevenh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would make that one last change to the internal run call, but it's a nit so approving anyway if you want to do in a follow up.

Copy link
Collaborator

@stevenh stevenh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@mdelapenya mdelapenya merged commit f954f6d into testcontainers:main Sep 30, 2024
118 checks passed
@mdelapenya mdelapenya deleted the etcd-module branch September 30, 2024 16:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New functionality or new behaviors on the existing one
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants