How to use AWS Secret manager to obtain credential for Confluent Kafka Cluster?

[lucas-silvs]

Hello, i try to create a ScaledObject to scaling with Kafka Consumer Group Lag, but i need to authenticate with the kafka Cluster (PLAIN Authentication). The idea is use the AWS Secret Manager to obtain the credentials of kafka cluster with the TriggerAuthentication, but not working yet. Is there any example with this same/equivalent scenario? Below is my Keda artifact:

Kubernetes Secret for AccessKey and AccessSecretKey:

apiVersion: v1
kind: Secret
metadata:
  name: keda-aws-credential-secret
  namespace: consumer
type: Opaque
data:
  AWS_ACCESS_KEY_ID: <base64-access-key>
  AWS_SECRET_ACCESS_KEY: <base64-key-access-key>

Secret Structure:

{
    "username":"username",
    "password":"password"
}

TriggerAuthentication:

apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
metadata:
  name: kafka-trigger-auth-aws-secret-manager
  namespace: consumer
spec:
  awsSecretManager:
    podIdentity:
      provider: aws
    credentials:                                     # Optional.
      accessKey:                                     # Required.
        valueFrom:                                   # Required.
          secretKeyRef:                              # Required.
            name: keda-aws-credential-secret  # Required.
            key: AWS_ACCESS_KEY_ID                  # Required.
      accessSecretKey:                               # Required.
        valueFrom:                                   # Required.
          secretKeyRef:                              # Required.
            name: keda-aws-credential-secret  # Required.
            key: AWS_SECRET_ACCESS_KEY
    region: us-east-1
    secrets:
      - parameter: password
        name: secret-name
      - parameter: username
        name: secret-name

ScaledObject:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: kafka-keda-consumer-trigger
  namespace: consumer
spec:
  minReplicaCount: 2  
  maxReplicaCount: 6 

  scaleTargetRef:
    name: consumer-deployment
  pollingInterval: 3
  triggers:
    - type: kafka
      metadata:
        bootstrapServers: <bootstraps-server>
        consumerGroup: consumer-teste
        topic: topic-name
        lagThreshold: "3"
        offsetResetPolicy: latest

        sasl: plaintext
      authenticationRef:
        name: kafka-trigger-auth-aws-secret-manager

Keda Operator Error Message:

2024-07-08T18:24:45Z	ERROR	scale_handler	error trying to read secret from Aws Secret Manager	{"type": "ScaledObject", "namespace": "consumer", "name": "kafka-keda-consumer-trigger", "triggerAuthRef.Name": "kafka-trigger-auth-aws-secret-manager", "secret.Name": "test", "secret.Version": "", "secret.VersionStage": "", "error": "operation error Secrets Manager: GetSecretValue, get identity: get credentials: failed to refresh cached credentials, no EC2 IMDS role found, operation error ec2imds: GetMetadata, exceeded maximum number of attempts, 3, request send failed, Get \"http://169.254.169.254/latest/meta-data/iam/security-credentials/\": dial tcp 169.254.169.254:80: i/o timeout"}
github.com/kedacore/keda/v2/pkg/scaling/resolver.resolveAuthRef
	/workspace/pkg/scaling/resolver/scale_resolvers.go:346
github.com/kedacore/keda/v2/pkg/scaling/resolver.ResolveAuthRefAndPodIdentity
	/workspace/pkg/scaling/resolver/scale_resolvers.go:183
github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).buildScalers.func1
	/workspace/pkg/scaling/scalers_builder.go:72
github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).buildScalers
	/workspace/pkg/scaling/scalers_builder.go:96
github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).performGetScalersCache
	/workspace/pkg/scaling/scale_handler.go:357
github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).GetScalersCache
	/workspace/pkg/scaling/scale_handler.go:282
github.com/kedacore/keda/v2/controllers/keda.(*ScaledObjectReconciler).getScaledObjectMetricSpecs
	/workspace/controllers/keda/hpa.go:217
github.com/kedacore/keda/v2/controllers/keda.(*ScaledObjectReconciler).newHPAForScaledObject
	/workspace/controllers/keda/hpa.go:72
github.com/kedacore/keda/v2/controllers/keda.(*ScaledObjectReconciler).createAndDeployNewHPA
	/workspace/controllers/keda/hpa.go:45
github.com/kedacore/keda/v2/controllers/keda.(*ScaledObjectReconciler).ensureHPAForScaledObjectExists
	/workspace/controllers/keda/scaledobject_controller.go:444
github.com/kedacore/keda/v2/controllers/keda.(*ScaledObjectReconciler).reconcileScaledObject
	/workspace/controllers/keda/scaledobject_controller.go:283
github.com/kedacore/keda/v2/controllers/keda.(*ScaledObjectReconciler).Reconcile
	/workspace/controllers/keda/scaledobject_controller.go:194
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
	/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:119
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
	/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:316
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227
2024-07-08T18:24:45Z	ERROR	scale_handler	error resolving auth params	{"type": "ScaledObject", "namespace": "consumer", "name": "kafka-keda-consumer-trigger", "triggerIndex": 0, "error": "error parsing kafka metadata: no username given"}
github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).buildScalers
	/workspace/pkg/scaling/scalers_builder.go:99
github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).performGetScalersCache
	/workspace/pkg/scaling/scale_handler.go:357
github.com/kedacore/keda/v2/pkg/scaling.(*scaleHandler).GetScalersCache
	/workspace/pkg/scaling/scale_handler.go:282
github.com/kedacore/keda/v2/controllers/keda.(*ScaledObjectReconciler).getScaledObjectMetricSpecs
	/workspace/controllers/keda/hpa.go:217
github.com/kedacore/keda/v2/controllers/keda.(*ScaledObjectReconciler).newHPAForScaledObject
	/workspace/controllers/keda/hpa.go:72
github.com/kedacore/keda/v2/controllers/keda.(*ScaledObjectReconciler).createAndDeployNewHPA
	/workspace/controllers/keda/hpa.go:45
github.com/kedacore/keda/v2/controllers/keda.(*ScaledObjectReconciler).ensureHPAForScaledObjectExists
	/workspace/controllers/keda/scaledobject_controller.go:444
github.com/kedacore/keda/v2/controllers/keda.(*ScaledObjectReconciler).reconcileScaledObject
	/workspace/controllers/keda/scaledobject_controller.go:283
github.com/kedacore/keda/v2/controllers/keda.(*ScaledObjectReconciler).Reconcile
	/workspace/controllers/keda/scaledobject_controller.go:194
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
	/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:119
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
	/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:316
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227
2024-07-08T18:24:45Z	ERROR	Error getting scalers	{"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "ScaledObject": {"name":"kafka-keda-consumer-trigger","namespace":"consumer"}, "namespace": "consumer", "name": "kafka-keda-consumer-trigger", "reconcileID": "02cd989f-84de-41ee-9ff9-290255bff7e6", "error": "error parsing kafka metadata: no username given"}