Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

spanner: support custom meterprovider for builtin metrics #11204

Open
tyrone-anz opened this issue Nov 29, 2024 · 2 comments
Open

spanner: support custom meterprovider for builtin metrics #11204

tyrone-anz opened this issue Nov 29, 2024 · 2 comments
Assignees
Labels
api: spanner Issues related to the Spanner API. triage me I really want to be triaged.

Comments

@tyrone-anz
Copy link

Is your feature request related to a problem? Please describe.

Spanner client is sending builtin/native metrics directly to GCP by default.

For services that have no permission, the metric export results to error log and span with following message.

Permission monitoring.timeSeries.create denied (or the resource may not exist).

For the above noise to go away, the native metrics can be disabled (SPANNER_DISABLE_BUILTIN_METRICS=true) or simply give service the missing permission. However, this is always something to keep in mind for a new service using spanner.

Describe the solution you'd like

Use the ClientConfig.OpenTelemetryMeterProvider from the config

Code affected.

var metricsProvider metric.MeterProvider
if emulatorAddr := os.Getenv("SPANNER_EMULATOR_HOST"); emulatorAddr != "" {
// Do not emit native metrics when emulator is being used
metricsProvider = noop.NewMeterProvider()
}
// Check if native metrics are disabled via env.
if disableNativeMetrics, _ := strconv.ParseBool(os.Getenv("SPANNER_DISABLE_BUILTIN_METRICS")); disableNativeMetrics {
config.DisableNativeMetrics = true
}
if config.DisableNativeMetrics {
// Do not emit native metrics when DisableNativeMetrics is set
metricsProvider = noop.NewMeterProvider()
}
// Create a OpenTelemetry metrics configuration
metricsTracerFactory, err := newBuiltinMetricsTracerFactory(ctx, database, metricsProvider)

Describe alternatives you've considered

  • Allow a param that sends builtin metrics to the global meter provider otel.GetMeterProvider()

Additional context

Add any other context or screenshots about the feature request here.

@tyrone-anz tyrone-anz added the triage me I really want to be triaged. label Nov 29, 2024
@product-auto-label product-auto-label bot added the api: spanner Issues related to the Spanner API. label Nov 29, 2024
@tyrone-anz
Copy link
Author

Just wanted to bump this one.
I am happy to raise PR for if okay to support.

@ymmt2005
Copy link

Hello,

We experienced the same issue. The issue was very confusing
because we configured a global meter provider with otel.SetMeterProvider.

It'd be great if the native metrics could use the global meter provider if set.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: spanner Issues related to the Spanner API. triage me I really want to be triaged.
Projects
None yet
Development

No branches or pull requests

3 participants