Issue with Prometheus ScaledObject - ERROR failed to ensure HPA is correctly created for ScaledObject

[Shyamsundar45]

I have a scaledobject with Prometheus trigger, I face below error and the error is similar to the issue mentioned #5093

2024-06-04T06:29:17Z	ERROR	Reconciler error	{"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "ScaledObject": {"name":"prom-scaledobject","namespace":"default"}, "namespace": "default", "name": "prom-scaledobject", "reconcileID": "b3c9e90e-cf16-484b-b275-04f67ebfcaff", "error": "HorizontalPodAutoscaler.autoscaling \"keda-hpa-prom-scaledobject\" is invalid: spec.metrics[0].external.target.averageValue: Invalid value: resource.Quantity{i:resource.int64Amount{value:0, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:\"0\", Format:\"DecimalSI\"}: must be positive"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
	/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:329
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227

Below is my manifest for keda scaledobject

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: prom-scaledobject
  namespace: default
spec:
  # This is the interval to check each trigger on. By default KEDA will check each trigger source on every ScaledObject every 30 seconds.
  pollingInterval: 30  # Optional. Default: 30 seconds
  cooldownPeriod: 120  # Optional. Default: 300 seconds

  minReplicaCount: 1     # Optional. Default: 0
  maxReplicaCount: 100
  idleReplicaCount: 0   # cannot be changed

  fallback:                                          # Optional. Section to specify fallback options
    failureThreshold: 3                              # Mandatory if fallback section is included
    replicas: 1

  advanced:
    restoreToOriginalReplicaCount: true

  scaleTargetRef:
    name: "{{ .Values.app }}-deployment"
    envSourceContainerName: "{{ .Values.app }}" # Optional. container-name

  triggers:
    - type: prometheus
      metadata:
        serverAddress: https://promurl/.test
        threshold: '1'
        activationThreshold: '5'
        query: sum(avg_over_time(rq_jobs{queue="default",namespace=“default”,status="queued”}[10m]))
        authModes: "basic"
      authenticationRef:
        name: keda-prom-creds

Am I doing something wrong in setting the values?

NOTE: All the variables from values file are also passed correctly.

Expected Behavior
The scaledobject should be able to create the HPA for prometheus trigger since all the values are already passed.

Actual Behavior
The HPA is not created so the scaledobject is not working as expected.

Steps to Reproduce the Problem
Create yaml manifest file for the ScaledObeject with trigger type Prometheus
Set the values as mentioned in the description
Run the manifest file to create the scaledobject

Logs from KEDA operator:

2024-06-04T06:29:17Z	ERROR	Reconciler error	{"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "ScaledObject": {"name":"prom-scaledobject","namespace":"default"}, "namespace": "default", "name": "prom-scaledobject", "reconcileID": "b3c9e90e-cf16-484b-b275-04f67ebfcaff", "error": "HorizontalPodAutoscaler.autoscaling \"keda-hpa-prom-scaledobject\" is invalid: spec.metrics[0].external.target.averageValue: Invalid value: resource.Quantity{i:resource.int64Amount{value:0, scale:0}, d:resource.infDecAmount{Dec:(*inf.Dec)(nil)}, s:\"0\", Format:\"DecimalSI\"}: must be positive"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
	/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:329
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
	/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:266
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
	/workspace/vendor/sigs.k8s.io/controller-runtime/pkg/internal/controller/controller.go:227

KEDA Version
2.13.1

Kubernetes Version
1.27

Platform
Google Cloud

Scaler Details
Prometheus

[goga1221]

I have the same problem, but with version 2.11, did you manage to solve it? @Shyamsundar45

[bajaj-hitesh]

I am getting the same error. Anyone able to solve this?

[guipace]

Also experiencing this. Could it be related to this long standing issue in Kubernetes?
kubernetes/kubernetes#87733