7.37.0
Prelude
Release on: 2022-06-27
- Please refer to the 7.37.0 tag on integrations-core for the list of changes on the Core Checks
Upgrade Notes
- OTLP ingest: Support for the deprecated
experimental.otlp
section and theDD_OTLP_GRPC_PORT
andDD_OTLP_HTTP_PORT
environment variables has been removed. Use theotlp_config
section or theDD_OTLP_CONFIG_RECEIVER_PROTOCOLS_GRPC_ENDPOINT
andDD_OTLP_CONFIG_RECEIVER_PROTOCOLS_HTTP_ENDPOINT
environment variables instead. - OTLP: Deprecated settings
otlp_config.metrics.report_quantiles
andotlp_config.metrics.send_monotonic_counter
have been removed in favor ofotlp_config.metrics.summaries.mode
andotlp_config.metrics.sums.cumulative_monotonic_mode
respectively.
New Features
- Adds User-level service unit filtering support for Journald log collection via
include_user_units
andexclude_user_units
. - A wildcard (*) can be used in either exclude_units or exclude_user_units if only a particular type of Journald log is desired.
- A new troubleshooting section has been added to the Agent CLI. This section will hold helpers to understand the Agent behavior. For now, the section only has two command to print the different metadata payloads sent by the Agent (v5 and inventory).
- APM: Incoming OTLP traces are now allowed to set their own sampling priority.
- Enable NPM NAT gateway lookup by default.
- Partial support of IPv6 on EKS clusters
- Fix the kubelet client when the IP of the host is IPv6.
- Fix the substitution of %%host%% patterns inside the auto-discovery annotations: If the concerned pod has an IPv6 and the %%host%% pattern appears inside an URL context, then the IPv6 is surrounded by square brackets.
- OTLP ingest now supports the same settings and protocol version as the OpenTelemetry Collector OTLP receiver v0.50.0.
- The Cloud Workload Security agent can now monitor and evaluate rules on bind syscall.
- [corechecks/snmp] add scale factor option to metric configurations
- Evaluate
memory.usage
metrics based on collected metrics.
Enhancement Notes
- APM:
DD_APM_FILTER_TAGS_REQUIRE
andDD_APM_FILTER_TAGS_REJECT
can now be a literal JSON array. e.g.["someKey:someValue"]
This allows for matching tag values with the space character in them. - SNMP Traps are now sent to a dedicated intake via the epforwarder.
- Update SNMP traps database to include integer enumerations.
- The Agent now supports a single
com.datadoghq.ad.checks
label in Docker, containerd, and Podman containers. It merges the contents of the existingcheck_names
,init_configs
(now optional), andinstances
annotations into a single JSON value. - Add a new Agent telemetry metric
autodiscovery_poll_duration
(histogram) to monitor configuration poll duration in Autodiscovery. - APM: Added
/config/set
endpoint in trace-agent to change configuration settings during runtime. Supports changing log level(log_level). - APM: When the X-Datadog-Trace-Count contains an invalid value, an error will be issued.
- Upgrade to Docker client 20.10, reducing the duration of docker check on Windows (requires Docker >= 20.10 on the host).
- The Agent maintains scheduled cluster and endpoint checks when the Cluster Agent is unavailable.
- The Cluster Agent followers now forward queries to the Cluster Agent leaders themselves. This allows a reduction in the overall number of connections to the Cluster Agent and better spreads the load between leader and forwarders.
- The
kube_namespace
tag is now included in all metrics, events, and service checks generated by the Helm check. - Include install_info to version-history.json
- Allow nightly builds install on non-prod repos
- Add a
kubernetes_node_annotations_as_tags
parameter to use Kubernetes node annotations as host tags. - Add more detailed logging around leadership status failures.
- Move the experimental SNMP Traps Listener configuration under
network_devices
. - Add support for the DNS Monitoring feature of NPM to Linux kernels older than 4.1.
- Adds
segment_name
andsegment_id
tags to PCF containers that belong to an isolation segment. - Make logs agent
additional_endpoints
reliable by default. This can be disabled by settingis_reliable: false
on the additional endpoint. - On Windows, if a
datadog.yaml
file is found during an installation or upgrade, the dialogs collecting the API Key and Site are skipped. - Resolve SNMP trap variables with integer enumerations to their string representation.
- [corechecks/snmp] Add profile
static_tags
config - Report telemetry metrics about the retry queue capacity:
datadog.agent.retry_queue_duration.capacity_secs
,datadog.agent.retry_queue_duration.bytes_per_sec
anddatadog.agent.retry_queue_duration.capacity_bytes
- Updated cloud providers to add the Instance ID as a host alias for EC2 instances, matching what other cloud providers do. This should help with correctly identifying hosts where the customer has changed the hostname to be different from the Instance ID.
- NTP check: Include
/etc/ntpd.conf
and/etc/openntpd/ntpd.conf
foruse_local_defined_servers
. - Kubernetes pod with short-lived containers do not have log lines duplicated with both container tags (the stopped one and the running one) when logs are collected. This feature is enabled by default, set
logs_config.validate_pod_container_id
tofalse
to disable it.
Security Notes
- The Agent is built with Go 1.17.11.
Bug Fixes
- Updates defaults for the port and binding host of the experimental traps listener.
- APM: The Agent is now performing rare span detection on all spans, as opposed to only dropped spans. This change will slightly reduce the number of rare spans kept unnecessarily.
- APM OTLP: This change ensures that the ingest now standardizes certain attribute keys to their correct Datadog tag counter parts, such as: container tags, "operation.name", "service.name", etc.
- APM: Fix a bug where the APM section of the GUI would not show up in older Internet Explorer versions on Windows.
- Support dynamic Auth Tokens in Kubernetes v1.22+ (Bound Service Account Token Volume).
- The
%%host%%
autodiscovery tag now works properly when using containerd, but only on Linux and when using IP v4 addresses. - Enhanced the coverage of pause-containers filtering on Containerd.
- APM: Fix the loss of trace metric container information when large payloads need to be split.
- Fix cri check producing no metrics when running on OpenShift / cri-o.
- Fix missing health status from Docker containers in Live Container View.
- Fix Agent startup failure when running as a non-privileged user (for instance, when running on OpenShift with
restricted
SCC). - Fix missing container metrics (container, containerd checks and live container view) on AWS Bottlerocket.
- APM: Fixed an issue where "CPU threshold exceeded" logs would show the wrong user CPU usage by a factor of 100.
- Ensures that when
kubernetes_namespace_labels_as_tags
is set, the namespace labels are always attached to metrics and logs, even when the pod is not ready yet. - Add missing support for UDPv6 receive path to NPM.
- The
agent workload-list --verbose
command and theworkload-list.log
file in the flare do not show containers' environment variables anymore. Except forDD_SERVICE
,DD_ENV
andDD_VERSION
. - Fixed a potential deadlock in the Python check runner during agent shutdown.
- Fixes issue where trace-agent would not report any version info.
- The DCA and the cluster runners no longer write warning logs to /tmp.
- Fixes an issue where the Agent would panic when trying to inspect Docker containers while the Docker daemon was unavailable or taking too long to respond.
Other Notes
- Exclude teradata on Mac agents.