Support scaling of Strimzi KafkaConnectors

[philipp94831]

KEDA allows auto-scaling of custom Kubernetes resources using the scale subresource. Strimzis KafkaConnector does have this resource but auto-scaling is failing with the following error: InvalidSelector: the HPA target's scale is missing a selector. According to this answer, the custom resource requires a status.selector field in the scale subresource, which is not present in the current CRDs. See also the linked blog post in the answer.

Is it possible to make auto-scaling of KafkaConnectors work although they don't create any pods inside Kubernetes?

These are examplary resources to reproduce the problem:

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaConnector
metadata:
  labels:
    strimzi.io/cluster: strimzi
  name: my-connector
spec:
  class: com.example.Connector
  config:
    topics: my-topic
    # configure the connector
  state: running
  tasksMax: 1

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: my-connector-so
spec:
  cooldownPeriod: 300
  maxReplicaCount: 2
  minReplicaCount: 1
  pollingInterval: 30
  scaleTargetRef:
    apiVersion: kafka.strimzi.io/v1beta2
    kind: KafkaConnector
    name: my-connector
  triggers:
  - metadata:
      bootstrapServers: strimzi-kafka-bootstrap:9092
      consumerGroup: connect-my-connector
      lagThreshold: "1000"
      offsetResetPolicy: earliest
      topic: my-topic
    type: kafka

Versions used: KEDA 2.15.1, Strimzi 0.41.0, Kubernetes 1.28

See also strimzi/strimzi-kafka-operator#10450

[JorTurFer]

I've already answered in strimzi repo. As we rely on the HPA controller, if the CRD doesn't support being scaled via HPA, we can't do anything