Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[EXPORTER] Export resource for prometheus #2301

Merged
merged 34 commits into from
Oct 3, 2023
Merged
Show file tree
Hide file tree
Changes from 30 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
dfefb44
Export resource for prometheus
owent Sep 8, 2023
ca3286a
Fix unit test in prometheus, fix resouce exporting for prometheus exp…
owent Sep 9, 2023
1436368
Fix constructor
owent Sep 9, 2023
22b8c94
Fix compiling problem
owent Sep 9, 2023
3b15502
Fix invalid resource pointer in unit test of prometheus
owent Sep 9, 2023
1ffba54
Fix "default member initializer" problem
owent Sep 12, 2023
401a8ac
Merge remote-tracking branch 'github/main' into emits_resource_for_pr…
owent Sep 14, 2023
d2e144a
Add `target_info` metric.
owent Sep 14, 2023
a30ddf7
Merge remote-tracking branch 'github/main' into emits_resource_for_pr…
owent Sep 14, 2023
02f4d04
Fix instance and job exporting
owent Sep 14, 2023
004a4d4
Merge remote-tracking branch 'github/main' into emits_resource_for_pr…
owent Sep 19, 2023
72ed751
Add options to let user to decide whether to populate target_info
owent Sep 19, 2023
00b0b66
Fix warnings
owent Sep 19, 2023
9f01fd4
Fix priority of conflict labels.
owent Sep 21, 2023
5d24d88
Fix unit test
owent Sep 21, 2023
e50a3d6
Allow some attributes in resource attributes but not in metric attrib…
owent Sep 21, 2023
88e9857
Change the info-typed target metric.
owent Sep 21, 2023
a0b322b
Do not ignore metric attributes.
owent Sep 21, 2023
33d3a47
Merge remote-tracking branch 'github/main' into emits_resource_for_pr…
owent Sep 23, 2023
0ab87c0
Merge remote-tracking branch 'github/main' into emits_resource_for_pr…
owent Sep 27, 2023
fcc397f
Fix unit tests after merged.
owent Sep 27, 2023
23f29cd
Merge remote-tracking branch 'github/main' into emits_resource_for_pr…
owent Sep 27, 2023
765eb99
Merge SanitizeName changes
owent Sep 27, 2023
c476939
Remove ignores in resource attributes
owent Sep 28, 2023
3ebe160
Merge remote-tracking branch 'opentelemetry/main'
owent Sep 29, 2023
422bca2
Remove labels conversations that should only be in target info.
owent Sep 29, 2023
d0f15f2
Remove unused variables.
owent Sep 29, 2023
bb62f0a
Populate target_info with every scrape
owent Oct 1, 2023
9826f67
Merge remote-tracking branch 'opentelemetry/main'
owent Oct 1, 2023
db783e2
Fix style
owent Oct 1, 2023
3517444
Merge remote-tracking branch 'opentelemetry/main'
owent Oct 3, 2023
dd737e3
Fix some legacy issues to remove `job` and `instance` labels.
owent Oct 3, 2023
b6ca2f2
Add scope for target-info
owent Oct 3, 2023
0149d5c
Restore `+2` when reserve space for vector.
owent Oct 3, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ Increment the:

* [DEPRECATION] Deprecate ZPAGES
[#2291](https://github.com/open-telemetry/opentelemetry-cpp/pull/2291)
* [EXPORTER] Prometheus exporter emit resource attributes
[#2301](https://github.com/open-telemetry/opentelemetry-cpp/pull/2301)
* [EXPORTER] Remove explicit timestamps from metric points exported by Prometheus
[#2324](https://github.com/open-telemetry/opentelemetry-cpp/pull/2324)
* [EXPORTER] Handle attribute key collisions caused by sanitation
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ class PrometheusCollector : public prometheus_client::Collectable
* This constructor initializes the collection for metrics to export
* in this class with default capacity
*/
explicit PrometheusCollector(sdk::metrics::MetricReader *reader);
explicit PrometheusCollector(sdk::metrics::MetricReader *reader, bool populate_target_info);

/**
* Collects all metrics data from metricsToCollect collection.
Expand All @@ -42,6 +42,7 @@ class PrometheusCollector : public prometheus_client::Collectable

private:
sdk::metrics::MetricReader *reader_;
bool populate_target_info_;

/*
* Lock when operating the metricsToCollect collection
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@ struct PrometheusExporterOptions

// The endpoint the Prometheus backend can collect metrics from
std::string url;

// Populating target_info
bool populate_target_info = true;
};

} // namespace metrics
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,12 +26,25 @@ class PrometheusExporterUtils
* to Prometheus metrics data collection
*
* @param records a collection of metrics in OpenTelemetry
* @param populate_target_info whether to populate target_info
* @return a collection of translated metrics that is acceptable by Prometheus
*/
static std::vector<::prometheus::MetricFamily> TranslateToPrometheus(
const sdk::metrics::ResourceMetrics &data);
const sdk::metrics::ResourceMetrics &data,
bool populate_target_info = true);

private:
/**
* Append key-value pair to prometheus labels.
*
* @param name label name
* @param value label value
* @param labels target labels
*/
static void AddPrometheusLabel(std::string name,
std::string value,
std::vector<::prometheus::ClientMetric::Label> *labels);

static opentelemetry::sdk::metrics::AggregationType getAggregationType(
const opentelemetry::sdk::metrics::PointType &point_type);

Expand All @@ -41,6 +54,12 @@ class PrometheusExporterUtils
static ::prometheus::MetricType TranslateType(opentelemetry::sdk::metrics::AggregationType kind,
bool is_monotonic = true);

/**
* Add a target_info metric to collect resource attributes
*/
static void SetTarget(const sdk::metrics::ResourceMetrics &data,
std::vector<::prometheus::MetricFamily> *output);

/**
* Set metric data for:
* Counter => Prometheus Counter
Expand All @@ -49,7 +68,8 @@ class PrometheusExporterUtils
static void SetData(std::vector<T> values,
const opentelemetry::sdk::metrics::PointAttributes &labels,
::prometheus::MetricType type,
::prometheus::MetricFamily *metric_family);
::prometheus::MetricFamily *metric_family,
const opentelemetry::sdk::resource::Resource *resource);

/**
* Set metric data for:
Expand All @@ -60,13 +80,15 @@ class PrometheusExporterUtils
const std::vector<double> &boundaries,
const std::vector<uint64_t> &counts,
const opentelemetry::sdk::metrics::PointAttributes &labels,
::prometheus::MetricFamily *metric_family);
::prometheus::MetricFamily *metric_family,
const opentelemetry::sdk::resource::Resource *resource);

/**
* Set time and labels to metric data
*/
static void SetMetricBasic(::prometheus::ClientMetric &metric,
const opentelemetry::sdk::metrics::PointAttributes &labels);
const opentelemetry::sdk::metrics::PointAttributes &labels,
const opentelemetry::sdk::resource::Resource *resource);

/**
* Convert attribute value to string
Expand Down
12 changes: 9 additions & 3 deletions exporters/prometheus/src/collector.cc
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,10 @@ namespace metrics
* This constructor initializes the collection for metrics to export
* in this class with default capacity
*/
PrometheusCollector::PrometheusCollector(sdk::metrics::MetricReader *reader) : reader_(reader) {}
PrometheusCollector::PrometheusCollector(sdk::metrics::MetricReader *reader,
bool populate_target_info)
: reader_(reader), populate_target_info_(populate_target_info)
{}

/**
* Collects all metrics data from metricsToCollect collection.
Expand All @@ -36,8 +39,11 @@ std::vector<prometheus_client::MetricFamily> PrometheusCollector::Collect() cons
collection_lock_.lock();

std::vector<prometheus_client::MetricFamily> result;
reader_->Collect([&result](sdk::metrics::ResourceMetrics &metric_data) {
auto prometheus_metric_data = PrometheusExporterUtils::TranslateToPrometheus(metric_data);

bool populate_target_info = populate_target_info_;
reader_->Collect([&result, populate_target_info](sdk::metrics::ResourceMetrics &metric_data) {
auto prometheus_metric_data =
PrometheusExporterUtils::TranslateToPrometheus(metric_data, populate_target_info);
marcalff marked this conversation as resolved.
Show resolved Hide resolved
for (auto &data : prometheus_metric_data)
result.emplace_back(data);
return true;
Expand Down
3 changes: 2 additions & 1 deletion exporters/prometheus/src/exporter.cc
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,8 @@ PrometheusExporter::PrometheusExporter(const PrometheusExporterOptions &options)
Shutdown(); // set MetricReader in shutdown state.
return;
}
collector_ = std::shared_ptr<PrometheusCollector>(new PrometheusCollector(this));
collector_ = std::shared_ptr<PrometheusCollector>(
new PrometheusCollector(this, options_.populate_target_info));

exposer_->RegisterCollectable(collector_);
}
Expand Down
102 changes: 84 additions & 18 deletions exporters/prometheus/src/exporter_utils.cc
Original file line number Diff line number Diff line change
@@ -1,14 +1,22 @@
// Copyright The OpenTelemetry Authors
// SPDX-License-Identifier: Apache-2.0

#include <limits>
#include <sstream>
#include <string>
#include <unordered_map>
#include <unordered_set>
#include <utility>
#include <vector>

#include "prometheus/metric_family.h"
#include "prometheus/metric_type.h"

#include <prometheus/metric_type.h>
#include "opentelemetry/exporters/prometheus/exporter_utils.h"
#include "opentelemetry/sdk/metrics/export/metric_producer.h"
#include "opentelemetry/sdk/resource/resource.h"
#include "opentelemetry/sdk/resource/semantic_conventions.h"
#include "opentelemetry/trace/semantic_conventions.h"

#include "opentelemetry/sdk/common/global_log_handler.h"

Expand All @@ -22,6 +30,7 @@ namespace metrics
{
namespace
{

/**
* Sanitize the given metric name by replacing invalid characters with _,
* ensuring that multiple consecutive _ characters are collapsed to a single _.
Expand Down Expand Up @@ -107,11 +116,25 @@ std::string SanitizeName(std::string name)
* @return a collection of translated metrics that is acceptable by Prometheus
*/
std::vector<prometheus_client::MetricFamily> PrometheusExporterUtils::TranslateToPrometheus(
const sdk::metrics::ResourceMetrics &data)
const sdk::metrics::ResourceMetrics &data,
bool populate_target_info)
{

// initialize output vector
std::size_t reserve_size = 1;
for (const auto &instrumentation_info : data.scope_metric_data_)
{
reserve_size += instrumentation_info.metric_data_.size();
}

std::vector<prometheus_client::MetricFamily> output;
output.reserve(reserve_size);

// Append target_info as the first metric
if (populate_target_info)
{
SetTarget(data, &output);
}

for (const auto &instrumentation_info : data.scope_metric_data_)
{
Expand Down Expand Up @@ -147,10 +170,10 @@ std::vector<prometheus_client::MetricFamily> PrometheusExporterUtils::TranslateT
}
else
{
sum = nostd::get<int64_t>(histogram_point_data.sum_);
sum = static_cast<double>(nostd::get<int64_t>(histogram_point_data.sum_));
Copy link
Member

@lalitb lalitb Oct 2, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not related to this PR, but there is possibility of precision loss while casting from int64_t to double for larger values ( > 2^52) of sum. Good to have a issue to track this (I will create one) - to check for any such loss and log a warning. No changes required in this PR.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I add this static cast because it will trigger a warning with clang 16 and -Werror will make it a error.
BTW:Do you think think it may have epsilon problem in the code metric->histogram.sample_count = static_cast<std::uint64_t>(values[1]); in PrometheusExporterUtils::SetValue ?

}
SetData(std::vector<double>{sum, (double)histogram_point_data.count_}, boundaries, counts,
point_data_attr.attributes, &metric_family);
point_data_attr.attributes, &metric_family, data.resource_);
}
else if (type == prometheus_client::MetricType::Gauge)
{
Expand All @@ -160,14 +183,14 @@ std::vector<prometheus_client::MetricFamily> PrometheusExporterUtils::TranslateT
auto last_value_point_data =
nostd::get<sdk::metrics::LastValuePointData>(point_data_attr.point_data);
std::vector<metric_sdk::ValueType> values{last_value_point_data.value_};
SetData(values, point_data_attr.attributes, type, &metric_family);
SetData(values, point_data_attr.attributes, type, &metric_family, data.resource_);
}
else if (nostd::holds_alternative<sdk::metrics::SumPointData>(point_data_attr.point_data))
{
auto sum_point_data =
nostd::get<sdk::metrics::SumPointData>(point_data_attr.point_data);
std::vector<metric_sdk::ValueType> values{sum_point_data.value_};
SetData(values, point_data_attr.attributes, type, &metric_family);
SetData(values, point_data_attr.attributes, type, &metric_family, data.resource_);
}
else
{
Expand All @@ -183,7 +206,7 @@ std::vector<prometheus_client::MetricFamily> PrometheusExporterUtils::TranslateT
auto sum_point_data =
nostd::get<sdk::metrics::SumPointData>(point_data_attr.point_data);
std::vector<metric_sdk::ValueType> values{sum_point_data.value_};
SetData(values, point_data_attr.attributes, type, &metric_family);
SetData(values, point_data_attr.attributes, type, &metric_family, data.resource_);
}
else
{
Expand All @@ -199,6 +222,17 @@ std::vector<prometheus_client::MetricFamily> PrometheusExporterUtils::TranslateT
return output;
}

void PrometheusExporterUtils::AddPrometheusLabel(
std::string name,
std::string value,
std::vector<::prometheus::ClientMetric::Label> *labels)
{
prometheus_client::ClientMetric::Label prometheus_label;
prometheus_label.name = std::move(name);
prometheus_label.value = std::move(value);
labels->emplace_back(std::move(prometheus_label));
}

metric_sdk::AggregationType PrometheusExporterUtils::getAggregationType(
const metric_sdk::PointType &point_type)
{
Expand Down Expand Up @@ -252,6 +286,35 @@ prometheus_client::MetricType PrometheusExporterUtils::TranslateType(
}
}

void PrometheusExporterUtils::SetTarget(const sdk::metrics::ResourceMetrics &data,
std::vector<::prometheus::MetricFamily> *output)
{
if (output == nullptr || data.resource_ == nullptr)
{
return;
}

prometheus_client::MetricFamily metric_family;
metric_family.name = "target";
metric_family.help = "Target metadata";
metric_family.type = prometheus_client::MetricType::Info;
metric_family.metric.emplace_back();

prometheus_client::ClientMetric &metric = metric_family.metric.back();
metric.info.value = 1.0;
lalitb marked this conversation as resolved.
Show resolved Hide resolved

metric_sdk::PointAttributes empty_attributes;
SetMetricBasic(metric, empty_attributes, data.resource_);

for (auto &label : data.resource_->GetAttributes())
{
AddPrometheusLabel(SanitizeName(label.first), AttributeValueToString(label.second),
&metric.label);
}

output->emplace_back(std::move(metric_family));
}

/**
* Set metric data for:
* sum => Prometheus Counter
Expand All @@ -260,11 +323,12 @@ template <typename T>
void PrometheusExporterUtils::SetData(std::vector<T> values,
const metric_sdk::PointAttributes &labels,
prometheus_client::MetricType type,
prometheus_client::MetricFamily *metric_family)
prometheus_client::MetricFamily *metric_family,
const opentelemetry::sdk::resource::Resource *resource)
{
metric_family->metric.emplace_back();
prometheus_client::ClientMetric &metric = metric_family->metric.back();
SetMetricBasic(metric, labels);
SetMetricBasic(metric, labels, resource);
SetValue(values, type, &metric);
}

Expand All @@ -277,21 +341,23 @@ void PrometheusExporterUtils::SetData(std::vector<T> values,
const std::vector<double> &boundaries,
const std::vector<uint64_t> &counts,
const metric_sdk::PointAttributes &labels,
prometheus_client::MetricFamily *metric_family)
prometheus_client::MetricFamily *metric_family,
const opentelemetry::sdk::resource::Resource *resource)
{
metric_family->metric.emplace_back();
prometheus_client::ClientMetric &metric = metric_family->metric.back();
SetMetricBasic(metric, labels);
SetMetricBasic(metric, labels, resource);
SetValue(values, boundaries, counts, &metric);
}

/**
* Set labels to metric data
*/
void PrometheusExporterUtils::SetMetricBasic(prometheus_client::ClientMetric &metric,
const metric_sdk::PointAttributes &labels)
const metric_sdk::PointAttributes &labels,
const opentelemetry::sdk::resource::Resource *resource)
{
if (labels.empty())
if (labels.empty() && nullptr == resource)
{
return;
}
Expand All @@ -300,7 +366,7 @@ void PrometheusExporterUtils::SetMetricBasic(prometheus_client::ClientMetric &me
// Note that attribute keys are sorted, but sanitized keys can be out-of-order.
// We could sort the sanitized keys again, but this seems too expensive to do
// in this hot code path. Instead, we ignore out-of-order keys and emit a warning.
metric.label.reserve(labels.size());
metric.label.reserve(labels.size() + 2);
owent marked this conversation as resolved.
Show resolved Hide resolved
std::string previous_key;
for (auto const &label : labels)
{
Expand Down Expand Up @@ -379,7 +445,7 @@ void PrometheusExporterUtils::SetValue(std::vector<T> values,
const auto &value_var = values[0];
if (nostd::holds_alternative<int64_t>(value_var))
{
value = nostd::get<int64_t>(value_var);
value = static_cast<double>(nostd::get<int64_t>(value_var));
}
else
{
Expand Down Expand Up @@ -414,9 +480,9 @@ void PrometheusExporterUtils::SetValue(std::vector<T> values,
const std::vector<uint64_t> &counts,
prometheus_client::ClientMetric *metric)
{
metric->histogram.sample_sum = values[0];
metric->histogram.sample_count = values[1];
int cumulative = 0;
metric->histogram.sample_sum = static_cast<double>(values[0]);
metric->histogram.sample_count = static_cast<std::uint64_t>(values[1]);
std::uint64_t cumulative = 0;
std::vector<prometheus_client::ClientMetric::Bucket> buckets;
uint32_t idx = 0;
for (const auto &boundary : boundaries)
Expand Down
4 changes: 2 additions & 2 deletions exporters/prometheus/test/collector_test.cc
Original file line number Diff line number Diff line change
Expand Up @@ -73,12 +73,12 @@ TEST(PrometheusCollector, BasicTests)
MockMetricReader *reader = new MockMetricReader();
MockMetricProducer *producer = new MockMetricProducer();
reader->SetMetricProducer(producer);
PrometheusCollector collector(reader);
PrometheusCollector collector(reader, true);
auto data = collector.Collect();

// Collection size should be the same as the size
// of the records collection produced by MetricProducer.
ASSERT_EQ(data.size(), 1);
ASSERT_EQ(data.size(), 2);
delete reader;
delete producer;
}
Loading