Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Logstash API spec - first pass #16546

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from
Draft

Conversation

karenzone
Copy link
Contributor

@karenzone karenzone added the docs label Oct 11, 2024
@karenzone karenzone self-assigned this Oct 11, 2024
Copy link
Contributor

📃 DOCS PREVIEWhttps://logstash_bk_16546.docs-preview.app.elstc.co/diff

Copy link
Contributor

📃 DOCS PREVIEWhttps://logstash_bk_16546.docs-preview.app.elstc.co/diff

Copy link
Contributor

📃 DOCS PREVIEWhttps://logstash_bk_16546.docs-preview.app.elstc.co/diff

Copy link
Contributor Author

@karenzone karenzone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lcawl, I left some questions and comments inline to discuss with you. Thanks for your help with this.

Comment on lines +40 to +42
- `pipelines`.
- `os`.
- `jvm`.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the trick to getting items on their own lines? (I tried \n, but they rendered literally.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Node Info section has subsections for pipelines, os, and jvm, each with their own examples.
ToDo: Format nested content

examples:
nodeInfoExample1:
pipelines:
- test:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trick to keep null from appearing for entries without values.

Comment on lines +111 to +118
- `jvm` gets JVM stats, including stats about threads, memory usage, garbage collectors, and uptime.
- `process` gets process stats, including stats about file descriptors, memory consumption, and CPU usage.
- `events` gets event-related statistics for the Logstash instance (regardless of how many pipelines were created and destroyed).
- `flow` gets flow-related statistics for the Logstash instance (regardless of how many pipelines were created and destroyed).
- `pipelines` gets runtime stats about each Logstash pipeline.
- `reloads` gets runtime stats about config reload successes and failures.
- `os` gets runtime stats about cgroups when Logstash is running in a container.
- `geoip_download_manager` gets stats for databases used with the Geoip filter plugin.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Formatting for bulleted list. Table format?

Comment on lines +134 to +152
nodeStatsExample1:
jvm:
threads:
count: 49
peak_count: 50
mem:
heap_used_percent: 14
heap_committed_in_bytes: 309866496
heap_max_in_bytes: 1037959168
heap_used_in_bytes: 151686096
non_heap_used_in_bytes: 122486176
non_heap_committed_in_bytes: 133222400
pools:
survivor:
peak_used_in_bytes: 8912896
used_in_bytes: 288776
peak_max_in_bytes: 35782656
max_in_bytes: 35782656
committed_in_bytes: 8912896
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Example is only partially reproduced.

Comment on lines +199 to +207
hot_threads:
- time: 2025-01-06T18:25:28-07:00
busiest_threads: 3
threads:
name: Ruby-0-Thread-7
percent_of_cpu_time: 0.0
state: timed_waiting
path: /path/to/logstash-8.17.0/vendor/bundle/jruby/1.9/gems/puma-2.16.0-java/lib/puma/thread_pool.rb:187
traces: java.lang.Object.wait(Native Method)", "org.jruby.RubyThread.sleep(RubyThread.java:1002)", "org.jruby.RubyKernel.sleep(RubyKernel.java:803)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the ordering principle for entries?
Note that busiest_threads is rendering last but it's not entered there.

Screenshot 2024-12-27 at 3 21 02 PM

path: /path/to/logstash-8.17.0/vendor/bundle/jruby/1.9/gems/puma-2.16.0-java/lib/puma/thread_pool.rb:187
traces: java.lang.Object.wait(Native Method)", "org.jruby.RubyThread.sleep(RubyThread.java:1002)", "org.jruby.RubyKernel.sleep(RubyKernel.java:803)

/_health_report:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Health_report appears last in the spec, but is rendering first. (Not sure it matters, but I'd like to know what's going on.)

Comment on lines +213 to +229
The health API returns a report with the health status of Logstash and the pipelines that are running inside of it.
The report contains a list of indicators that compose Logstash functionality.

Each indicator has a health status of: green, unknown, yellow, or red.
The indicator provides an explanation and metadata describing the reason for its current health status.

The top-level status is controlled by the worst indicator status.

In the event that an indicator status is non-green, a list of impacts may be present in the indicator result which detail the functionalities that are negatively affected by the health issue.
Each impact carries with it a severity level, an area of the system that is affected, and a simple description of the impact on the system.

Some health indicators can determine the root cause of a health problem and prescribe a set of steps that can be performed in order to improve the health of the system.
The root cause and remediation steps are encapsulated in a diagnosis.
A diagnosis contains a cause detailing a root cause analysis, an action containing a brief description of the steps to take to fix the problem, and the URL for detailed troubleshooting help.

NOTE: The health indicators perform root cause analysis of non-green health statuses.
This can be computationally expensive when called frequently.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This content needs line breaks. I tried \n, but they render literally.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe this is just pure YAML. The scalar (>) folds on line 212 (description: >). If instead we use a literal scalar | it should preserve line breaks exactly as written (description: |). https://github.com/elastic/logstash/pull/16546/files#r1899659257

Comment on lines +246 to +251
status:
- green: Logstash is healthy.
unknown: Logstash health could not be determined.
yellow: The functionality of Logstash is in a degraded state and may need remediation to avoid the health becoming red.
red: Logstash is experiencing an outage or certain features are unavailable for use.
indicators: Information about the health of Logstash indicators.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Health report API docs show hierarchical descriptions rather that examples. What's the best way to represent this type of content? Also, research nested format.

/_health_report:
get:
summary: Get health status
description: >
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
description: >
description: |

Use literal scalar instead of folding to preserve newlines.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants