Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doc: Add json_lines known issue to release notes #16831

Merged
merged 7 commits into from
Dec 26, 2024
Merged

Doc: Add json_lines known issue to release notes #16831

Changes from 1 commit
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
e2068fb
Doc: Add json_lines known issue to release notes
karenzone Dec 20, 2024
9302fc0
Incorporate review feedback
karenzone Dec 20, 2024
0a3e338
Remove extra returns
karenzone Dec 20, 2024
ef57577
Update docs/static/releasenotes.asciidoc
karenzone Dec 20, 2024
ed09abf
Improve content and add known issue to other releases
karenzone Dec 21, 2024
0ba13aa
Correct referenced plugin
karenzone Dec 23, 2024
7fe9d44
Pin example link to static file
karenzone Dec 23, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 36 additions & 5 deletions docs/static/releasenotes.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,12 +76,43 @@ This section summarizes the changes in the following releases:
=== Logstash 8.17.0 Release Notes

[[known-issues-8-17-0]]
==== Known issue
==== Known issues

[[known-issue-8-17-0-jvm]]
===== JVM version changes needed when upgrading Logstash from 8.12.0 (or earlier)

If the `jvm.options` file was modified and not overwritten with the newest version, you may see a "deserialize invocation error" message, causing the pipeline to crash.
Users are affected if the Persistent Queue (PQ) is enabled, and the pipeline is processing messages larger than 20MB.

**Solution:** Apply the default change contained in the newer 'jvm.options' file, as seen in this https://github.com/elastic/logstash/blob/main/config/jvm.options#L66-L82[example].
karenzone marked this conversation as resolved.
Show resolved Hide resolved

[[known-issue-8-17-0-json_lines]]
===== Error with `json_lines` codec 3.2.0 (or earlier) and {ls} 8.17.1 (or earlier)
Copy link
Contributor

@mashhurs mashhurs Dec 20, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
===== Error with `json_lines` codec 3.2.0 (or earlier) and {ls} 8.17.1 (or earlier)
===== Error with `json_lines` codec 3.2.0 (or later) with {ls} 8.17.0

and {ls} 8.17.1 (or earlier) - part is a bit confusing to me. I support removing LS version from this title which brings reader to "... present in {ls} versions 8.16.0, 8.16.1, and 8.17.0." and to mitigation section.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was based on description in issue: "When using the json_lines codec > 3.2.0 with versions of Logstash < 8.17.1, the following error may be emitted, crashing the pipelines"

If that's not accurate, I'll update it here, and it will also need to be updated in the issue.

Copy link
Member

@robbavey robbavey Dec 20, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about using the symptom as the heading, rather than json_lines codec?

===== "Input Buffer Full" error With Logstash `8.17.0`, `8.16.1` and `8.16.0`

Then describe the instances of what happens when this error occurs, how this would happen and how to mitigate, with Logstash->Logstash communication first, as it is the most likely, then the json_lines case?

Leading with json_lines isn't necessarily obvious as the codec is used "silently" by the Logstash integration plugin.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
===== Error with `json_lines` codec 3.2.0 (or earlier) and {ls} 8.17.1 (or earlier)
===== Error with `json_lines` codec 3.2.0 (or later) and {ls} 8.17.1 (or earlier)

Based on description in the issue, I think this should be 3.2.0 or later for json_lines?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, json_lines codec >= 3.2.0 versions which have an ability to set the buffer limit.


This problem is most likely to be seen when you are using the Logstash integration plugin to ship data between two {ls} instances, but may appear in other situations, too.
An error similar to this one may be emitted, crashing the pipelines:

```
unable to process event. {:message=>"input buffer full", :class=>"Java::JavaLang::IllegalStateException", :backtrace=>["org.logstash.common.BufferedTokenizerExt.extract(BufferedTokenizerExt.java:83)", "usr.share.logstash.vendor.bundle.jruby.$3_dot_1_dot_0.gems.logstash_minus_codec_minus_json_lines_minus_3_dot_2_dot_2.lib.logstash.codecs.json_lines.RUBY$method$decode$0(/usr/share/logstash/vendor/bundle/jruby/3.1.0/gems/logstash-codec-json_lines-3.2.2/lib/logstash/codecs/json_lines.rb:69)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:165)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:185)",
```

The issue was fixed in https://github.com/elastic/logstash/pull/16760, but is likely present in {ls} versions `8.16.0`, `8.16.1`, and `8.17.0`.

**Workaround for {ls}-to-{ls} communication**

This issue can be mitigated by:

* Downgrading the _receiving_ Logstash to `8.16.2`, or any Logstash in the `8.15` series, **_AND/OR_**
* Upgrading the logstash integration filter of the _sending_ Logstash to version `1.0.4`.

**Workaround for other `json_lines` codec situations**

This issue can be mitigated by:

* Downgrading Logstash to `8.16.2`, or any Logstash in the `8.15` series.



When upgrading Logstash to 8.12.0 or earlier, if the `jvm.options` was previously modified and not overwritten with the newest version,
users may see a "deserialize invocation error" message causing the pipeline to crash. Users are affected if the Persistent Queue
is enabled and the pipeline is processing messages larger than 20MB.
The solution is to apply the default change contained in the newer 'jvm.options' file, as seen https://github.com/elastic/logstash/blob/main/config/jvm.options#L66-L82[here].

[[notable-8-17-0]]
==== Notable fixes and improvements
Expand Down
Loading