Skip to content

Commit

Permalink
fix: aligning the interceptor metrics vars with the OTEL spec
Browse files Browse the repository at this point in the history
Signed-off-by: Joe Wogan <[email protected]>
  • Loading branch information
zorocloud committed May 24, 2024
1 parent b283195 commit d4aa1dc
Show file tree
Hide file tree
Showing 6 changed files with 22 additions and 20 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ This changelog keeps track of work items that have been completed and are ready

### Fixes

- **General**: Align the interceptor metrics env var configuration with the OTEL spec ([#1031](https://github.com/kedacore/http-add-on/issues/1031))
- **General**: TODO ([#TODO](https://github.com/kedacore/http-add-on/issues/TODO))

### Deprecations
Expand Down
12 changes: 6 additions & 6 deletions config/interceptor/e2e-test/otel/deployment.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,15 @@ spec:
containers:
- name: interceptor
env:
- name: KEDA_HTTP_OTEL_PROM_EXPORTER_ENABLED
- name: OTEL_PROM_EXPORTER_ENABLED
value: "true"
- name: KEDA_HTTP_OTEL_PROM_EXPORTER_PORT
- name: OTEL_PROM_EXPORTER_PORT
value: "2223"
- name: KEDA_HTTP_OTEL_HTTP_EXPORTER_ENABLED
- name: OTEL_EXPORTER_OTLP_METRICS_ENABLED
value: "true"
- name: KEDA_HTTP_OTEL_HTTP_COLLECTOR_ENDPOINT
- name: OTEL_EXPORTER_OTLP_METRICS_ENDPOINT
value: "opentelemetry-collector.open-telemetry-system:4318"
- name: KEDA_HTTP_OTEL_HTTP_COLLECTOR_INSECURE
- name: OTEL_EXPORTER_OTLP_METRICS_INSECURE
value: "true"
- name: KEDA_HTTP_OTEL_METRIC_EXPORT_INTERVAL
- name: OTEL_EXPORTER_OTLP_METRICS_EXPORT_INTERVAL
value: "1"
8 changes: 4 additions & 4 deletions docs/operate.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,16 +9,16 @@ There are currently 2 supported methods for exposing metrics from the intercepto
### Configuring the Prometheus compatible metrics endpoint
When configured, the interceptor proxy can expose metrics on a Prometheus compatible endpoint.

This endpoint can be enabled by setting the `KEDA_HTTP_OTEL_PROM_EXPORTER_ENABLED` environment variable to `true` on the interceptor deployment (`true` by default) and by setting `KEDA_HTTP_OTEL_PROM_EXPORTER_PORT` to an unused port for the endpoint to be made avaialble on (`2223` by default).
This endpoint can be enabled by setting the `OTEL_PROM_EXPORTER_ENABLED` environment variable to `true` on the interceptor deployment (`true` by default) and by setting `OTEL_PROM_EXPORTER_PORT` to an unused port for the endpoint to be made avaialble on (`2223` by default).

### Configuring the OTEL HTTP exporter
When configured, the interceptor proxy can export metrics to a OTEL HTTP collector.

The OTEL exporter can be enabled by setting the `KEDA_HTTP_OTEL_HTTP_EXPORTER_ENABLED` environment variable to `true` on the interceptor deployment (`false` by default). When enabled the `KEDA_HTTP_OTEL_HTTP_COLLECTOR_ENDPOINT` environment variable must also be configured so the exporter knows what collector to send the metrics to (e.g. opentelemetry-collector.open-telemetry-system:4318).
The OTEL exporter can be enabled by setting the `OTEL_EXPORTER_OTLP_METRICS_ENABLED` environment variable to `true` on the interceptor deployment (`false` by default). When enabled the `OTEL_EXPORTER_OTLP_METRICS_ENDPOINT` environment variable must also be configured so the exporter knows what collector to send the metrics to (e.g. opentelemetry-collector.open-telemetry-system:4318).

If the collector is exposed on a unsecured endpoint then you can set the `KEDA_HTTP_OTEL_HTTP_COLLECTOR_INSECURE` environment variable to `true` (`false` by default) which will disable client security on the exporter.
If the collector is exposed on a unsecured endpoint then you can set the `OTEL_EXPORTER_OTLP_METRICS_INSECURE` environment variable to `true` (`false` by default) which will disable client security on the exporter.

If you need to provide any headers such as authentication details in order to utilise your OTEL collector you can add them into the `KEDA_HTTP_OTEL_HTTP_HEADERS` environment variable. The frequency at which the metrics are exported can be configured by setting `KEDA_HTTP_OTEL_METRIC_EXPORT_INTERVAL` to the number of seconds you require between each export interval (`30` by default).
If you need to provide any headers such as authentication details in order to utilise your OTEL collector you can add them into the `OTEL_EXPORTER_OTLP_METRICS_HEADERS` environment variable. The frequency at which the metrics are exported can be configured by setting `OTEL_EXPORTER_OTLP_METRICS_EXPORT_INTERVAL` to the number of seconds you require between each export interval (`30` by default).

# Configuring TLS for the KEDA HTTP Add-on interceptor proxy

Expand Down
14 changes: 7 additions & 7 deletions interceptor/config/metrics.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,19 +7,19 @@ import (
// Metrics is the configuration for configuring metrics in the interceptor.
type Metrics struct {
// Sets whether or not to enable the Prometheus metrics exporter
OtelPrometheusExporterEnabled bool `envconfig:"KEDA_HTTP_OTEL_PROM_EXPORTER_ENABLED" default:"true"`
OtelPrometheusExporterEnabled bool `envconfig:"OTEL_PROM_EXPORTER_ENABLED" default:"true"`
// Sets the port which the Prometheus compatible metrics endpoint should be served on
OtelPrometheusExporterPort int `envconfig:"KEDA_HTTP_OTEL_PROM_EXPORTER_PORT" default:"2223"`
OtelPrometheusExporterPort int `envconfig:"OTEL_PROM_EXPORTER_PORT" default:"2223"`
// Sets whether or not to enable the OTEL metrics exporter
OtelHTTPExporterEnabled bool `envconfig:"KEDA_HTTP_OTEL_HTTP_EXPORTER_ENABLED" default:"false"`
OtelHTTPExporterEnabled bool `envconfig:"OTEL_EXPORTER_OTLP_METRICS_ENABLED" default:"false"`
// Sets the HTTP endpoint where metrics should be sent to
OtelHTTPCollectorEndpoint string `envconfig:"KEDA_HTTP_OTEL_HTTP_COLLECTOR_ENDPOINT" default:"localhost:4318"`
OtelHTTPCollectorEndpoint string `envconfig:"OTEL_EXPORTER_OTLP_METRICS_ENDPOINT" default:"localhost:4318"`
// Sets the OTLP headers required by the otel exporter
OtelHTTPHeaders string `envconfig:"KEDA_HTTP_OTEL_HTTP_HEADERS" default:""`
OtelHTTPHeaders string `envconfig:"OTEL_EXPORTER_OTLP_METRICS_HEADERS" default:""`
// Set the connection to the otel HTTP collector endpoint to use HTTP rather than HTTPS
OtelHTTPCollectorInsecure bool `envconfig:"KEDA_HTTP_OTEL_HTTP_COLLECTOR_INSECURE" default:"false"`
OtelHTTPCollectorInsecure bool `envconfig:"OTEL_EXPORTER_OTLP_METRICS_INSECURE" default:"false"`
// Set the interval in seconds to export otel metrics to the configured collector endpoint
OtelMetricExportInterval int `envconfig:"KEDA_HTTP_OTEL_METRIC_EXPORT_INTERVAL" default:"30"`
OtelMetricExportInterval int `envconfig:"OTEL_EXPORTER_OTLP_METRICS_EXPORT_INTERVAL" default:"30"`
}

// Parse parses standard configs using envconfig and returns a pointer to the
Expand Down
5 changes: 3 additions & 2 deletions interceptor/metrics/otelmetrics.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,12 +30,13 @@ func NewOtelMetrics(metricsConfig *config.Metrics, options ...metric.Option) *Ot
endpoint := otlpmetrichttp.WithEndpoint(metricsConfig.OtelHTTPCollectorEndpoint)
headersFromEnvVar := getHeaders(metricsConfig.OtelHTTPHeaders)
headers := otlpmetrichttp.WithHeaders(headersFromEnvVar)
path := otlpmetrichttp.WithURLPath("/v1/metrics")

if metricsConfig.OtelHTTPCollectorInsecure {
insecure := otlpmetrichttp.WithInsecure()
exporter, err = otlpmetrichttp.New(ctx, endpoint, headers, insecure)
exporter, err = otlpmetrichttp.New(ctx, endpoint, headers, path, insecure)
} else {
exporter, err = otlpmetrichttp.New(ctx, endpoint, headers)
exporter, err = otlpmetrichttp.New(ctx, endpoint, headers, path)
}

if err != nil {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -171,7 +171,7 @@ func TestMetricGeneration(t *testing.T) {
// If the metric is not found first time around then retry with a delay.
if !ok {
// Add a small sleep to allow metrics to be pushed from the exporter to the collector
time.Sleep(5 * time.Second)
time.Sleep(10 * time.Second)
// Fetch metrics and validate them
family := fetchAndParsePrometheusMetrics(t, fmt.Sprintf("curl --insecure %s", otelCollectorPromURL))
val, ok = family["interceptor_request_count_total"]
Expand Down

0 comments on commit d4aa1dc

Please sign in to comment.