Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add reference to prefer-ilm setting for apm lifeycle known bug #4501

Merged
merged 6 commits into from
Nov 8, 2024
Merged

Add reference to prefer-ilm setting for apm lifeycle known bug #4501

Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
f2eec04
Add reference to prefer-ilm setting for apm lifeycle known bug
lahsivjar Nov 8, 2024
b6e7c70
Refactor
lahsivjar Nov 8, 2024
3de4736
APM known issue correction
inge4pres Nov 8, 2024
d226ee9
Update docs/en/observability/apm/known-issues.asciidoc
bmorelli25 Nov 8, 2024
ef8f699
clean up text
bmorelli25 Nov 8, 2024
37b338c
Update docs/en/observability/apm/known-issues.asciidoc
bmorelli25 Nov 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 16 additions & 52 deletions docs/en/observability/apm/known-issues.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,11 @@ _Versions: XX.XX.XX, YY.YY.YY, ZZ.ZZ.ZZ_
[discrete]
== Upgrading to v8.15.x may cause ingestion to fail

_Elastic Stack versions: 8.15.0+_
_Elastic Stack versions: 8.15.0, 8.15.1, 8.15.2, 8.15.3_ +
_Fixed in Elastic Stack version 8.15.4_

// The conditions in which this issue occurs
The issue only occurs when _upgrading_ the {stack} from 8.12.2 or lower directly to any 8.15.x version.
The issue only occurs when _upgrading_ the {stack} from 8.12.2 or lower directly to any 8.15.x version prior to 8.15.4.
The issue does _not_ occur when creating a _new_ cluster using any 8.15.x version, or when upgrading
from 8.12.2 to 8.13.x or 8.14.x and then to 8.15.x.

Expand Down Expand Up @@ -84,63 +85,26 @@ stream lifecycle configuration, such existing data streams become unmanaged for
lifecycle configurations.

// How to fix it
Upgrading to 8.15.1 should fix any new indices created for the data stream. However,
indices created in version 8.15.0 would remain unmanaged if the default ILM policy is
used. One of the following approaches can be adopted to fix the unmanaged indices:

. Manually delete the indices when they are no longer needed.
. Explicitly configure APM data streams with the default data stream lifecycle config.
Using this approach would migrate all data streams to use data stream lifecycles,
which should be equivalent to the default ILM policies:
Upgrading to 8.15.1 resolves the lifecycle issue for any new indices created for APM data streams.
However, indices created in version 8.15.0 will remain unmanaged if the default ILM policy is in place.
To fix these unmanaged indices, consider one of the following approaches:

. Manually delete the unmanaged indices when they are no longer needed.
. Explicitly configure APM data streams to use the default data stream lifecycle configuration.
This approach migrates all affected data streams to use data stream lifecycles,
maintaining behavior equivalent to the default ILM policies.
Apply this fix only to data streams that have unmanaged indices due to missing default ILM policies.
+
[source,txt]
----
PUT _data_stream/traces-apm-*/_lifecycle
PUT _data_stream/{{data_stream_name}}-{{data_stream_namespace}}/_lifecycle
{
"data_retention": "10d"
}

PUT _data_stream/traces-apm.rum*/_lifecycle
{
"data_retention": "90d"
}

PUT _data_stream/traces-apm.sampled*/_lifecycle
{
"data_retention": "1h"
}

PUT _data_stream/metrics-apm.*.1m-*/_lifecycle
{
"data_retention": "90d"
}

PUT _data_stream/metrics-apm.*.10m-*/_lifecycle
{
"data_retention": "180d"
}

PUT _data_stream/metrics-apm.*.60m-*/_lifecycle
{
"data_retention": "390d"
}

PUT _data_stream/metrics-apm.internal-*/_lifecycle
{
"data_retention": "90d"
}

PUT _data_stream/metrics-apm.app.*/_lifecycle
{
"data_retention": "90d"
}

PUT _data_stream/logs-apm.*/_lifecycle
{
"data_retention": "10d"
"data_retention": <data_retention_period>
}
----

Default `<data_retention_period>` for each data stream is available in {observability-guide}/apm-ilm-how-to.html[this guide].

// Link to fix if it exists
This issue is fixed in 8.15.1 (https://github.com/elastic/elasticsearch/pull/112432[elastic/elasticsearch#112432]).

Expand Down