Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hotfixes (histogram buckets + hex name + engine beholder metrics) #15626

Merged
merged 3 commits into from
Dec 11, 2024
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
wiring engine histogram buckets through to beholder Client (#15508)
* wiring engine histogram buckets through to beholder Client

* bumping common

* Move metric views (#15515)

* bumping common

---------

Co-authored-by: Pavel <177363085+pkcll@users.noreply.github.com>
  • Loading branch information
2 people authored and bolekk committed Dec 11, 2024
commit 31cb374fd40e22d2570fcb51b3cd3d98b84da544
5 changes: 5 additions & 0 deletions core/cmd/shell.go
Original file line number Diff line number Diff line change
@@ -51,6 +51,7 @@ import (
"github.com/smartcontractkit/chainlink/v2/core/services/relay/evm/mercury/wsrpc/cache"
"github.com/smartcontractkit/chainlink/v2/core/services/versioning"
"github.com/smartcontractkit/chainlink/v2/core/services/webhook"
"github.com/smartcontractkit/chainlink/v2/core/services/workflows"
"github.com/smartcontractkit/chainlink/v2/core/sessions"
"github.com/smartcontractkit/chainlink/v2/core/static"
"github.com/smartcontractkit/chainlink/v2/core/store/migrate"
@@ -109,6 +110,10 @@ func initGlobals(cfgProm config.Prometheus, cfgTracing config.Tracing, cfgTeleme
AuthPublicKeyHex: csaPubKeyHex,
AuthHeaders: beholderAuthHeaders,
}
// note: due to the OTEL specification, all histogram buckets
// must be defined when the beholder client is created
clientCfg.MetricViews = append(clientCfg.MetricViews, workflows.MetricViews()...)

if tracingCfg.Enabled {
clientCfg.TraceSpanExporter, err = tracingCfg.NewSpanExporter()
if err != nil {
32 changes: 32 additions & 0 deletions core/services/workflows/monitoring.go
Original file line number Diff line number Diff line change
@@ -5,6 +5,7 @@ import (
"fmt"

"go.opentelemetry.io/otel/metric"
sdkmetric "go.opentelemetry.io/otel/sdk/metric"

"github.com/smartcontractkit/chainlink-common/pkg/beholder"
"github.com/smartcontractkit/chainlink-common/pkg/metrics"
@@ -53,6 +54,37 @@ func initMonitoringResources() (err error) {
return nil
}

// Note: due to the OTEL specification, all histogram buckets
// Must be defined when the beholder client is created
func MetricViews() []sdkmetric.View {
return []sdkmetric.View{
sdkmetric.NewView(
sdkmetric.Instrument{Name: "platform_engine_workflow_earlyexit_time_seconds"},
sdkmetric.Stream{Aggregation: sdkmetric.AggregationExplicitBucketHistogram{
Boundaries: []float64{0, 1, 10, 100},
}},
),
sdkmetric.NewView(
sdkmetric.Instrument{Name: "platform_engine_workflow_completed_time_seconds"},
sdkmetric.Stream{Aggregation: sdkmetric.AggregationExplicitBucketHistogram{
Boundaries: []float64{0, 100, 1000, 10_000, 50_000, 100_0000, 500_000},
}},
),
sdkmetric.NewView(
sdkmetric.Instrument{Name: "platform_engine_workflow_error_time_seconds"},
sdkmetric.Stream{Aggregation: sdkmetric.AggregationExplicitBucketHistogram{
Boundaries: []float64{0, 20, 60, 120, 240},
}},
),
sdkmetric.NewView(
sdkmetric.Instrument{Name: "platform_engine_workflow_step_time_seconds"},
sdkmetric.Stream{Aggregation: sdkmetric.AggregationExplicitBucketHistogram{
Boundaries: []float64{0, 20, 60, 120, 240},
}},
),
}
}

// workflowsMetricLabeler wraps monitoring.MetricsLabeler to provide workflow specific utilities
// for monitoring resources
type workflowsMetricLabeler struct {
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
@@ -104,6 +104,7 @@ require (
go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin v0.49.0
go.opentelemetry.io/otel v1.31.0
go.opentelemetry.io/otel/metric v1.31.0
go.opentelemetry.io/otel/sdk/metric v1.31.0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this didn't come with a corresponding go.sum change?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No because it's still included, just sorted differently.

go.opentelemetry.io/otel/trace v1.31.0
go.uber.org/multierr v1.11.0
go.uber.org/zap v1.27.0
@@ -361,7 +362,6 @@ require (
go.opentelemetry.io/otel/log v0.6.0 // indirect
go.opentelemetry.io/otel/sdk v1.31.0 // indirect
go.opentelemetry.io/otel/sdk/log v0.6.0 // indirect
go.opentelemetry.io/otel/sdk/metric v1.31.0 // indirect
go.opentelemetry.io/proto/otlp v1.3.1 // indirect
go.uber.org/ratelimit v0.3.0 // indirect
golang.org/x/arch v0.11.0 // indirect