-
Notifications
You must be signed in to change notification settings - Fork 3.5k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge branch '8.11' into testing-logstash-chainguard
* 8.11: [ci] Add testing phase to exhaustive tests suite (#15711) (#15717) [ci] Reusable unit + IT test steps for Buildkite (#15708) (#15714) Remove temporary dependencies in logstash-core.gemspec (#15694) Release notes for 8.11.3 (#15681) bump version to 8.11.4 (#15682) Doc: Add docs for extending integrations with filter-elastic_integrations (#15518) (#15675) Update JRuby to 9.4.5.0 (#15531) (#15670) bump to 8.11.3 (#15669) Release notes for 8.11.2 (#15660) Doc: Update Logstash intro and security overview for serverless (#15313) (#15664) Shutdown DLQ segments flusher only if it has been started (#15649) (#15656)
- Loading branch information
Showing
11 changed files
with
207 additions
and
61 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,79 @@ | ||
| [[ea-integrations]] | ||
| == Using {ls} with Elastic {integrations} (Beta) | ||
|
|
||
| You can take advantage of the extensive, built-in capabilities of Elastic {integrations}--such as managing data collection, transformation, and visualization--and then use {ls} for additional data processing and output options. | ||
| {ls} can further expand capabilities for use cases where you need additional processing, or if you need your data delivered to multiple destinations. | ||
|
|
||
| [discrete] | ||
| [[integrations-value]] | ||
| === Elastic {integrations}: ingesting to visualizing | ||
|
|
||
| https://docs.elastic.co/integrations[Elastic {integrations}] provide quick, end-to-end solutions for: | ||
|
|
||
| * ingesting data from a variety of data sources | ||
| * getting the data into the {stack}, and | ||
| * visualizing it with purpose-built dashboards. | ||
|
|
||
| {integrations} are available for https://docs.elastic.co/integrations/all_integrations[popular services and platforms], such as Nginx, AWS, and MongoDB, as well as many generic input types like log files. | ||
| Each integration includes pre-packaged assets to help reduce the time between ingest and insights. | ||
|
|
||
| To see available integrations, go to the {kib} home page, and click **Add {integrations}**. | ||
| You can use the query bar to search for integrations you may want to use. | ||
| When you find an integration for your data source, the UI walks you through adding and configuring it. | ||
|
|
||
| [discrete] | ||
| [[integrations-and-ls]] | ||
| === Extend {integrations} with {ls} (Beta) | ||
|
|
||
| Logstash can run the ingest pipeline component of your integration when you use the Logstash filter-elastic_integration plugin. | ||
|
|
||
| .How to | ||
|
|
||
| **** | ||
| Create a {ls} pipeline that uses the <<plugins-inputs-elastic_agent,elastic_agent input>> plugin, and the https://github.com/elastic/logstash-filter-elastic_integration[logstash-filter-elastic_integration] plugin as the _first_ filter in your {ls} pipeline. | ||
| You can add more filters for additional processing, but they must come after the `logstash-filter-elastic_integration` plugin in your configuration. | ||
| Add an output plugin to complete your pipeline. | ||
| **** | ||
|
|
||
|
|
||
| **Sample pipeline configuration** | ||
|
|
||
| [source,ruby] | ||
| ----- | ||
| input { | ||
| elastic_agent { | ||
| port => 5044 | ||
| } | ||
| } | ||
| filter { | ||
| elastic_integration{ <1> | ||
| cloud_id => "<cloud id>" | ||
| cloud_auth => "<your_cloud-auth" | ||
| } | ||
| translate { <2> | ||
| source => "[http][host]" | ||
| target => "[@metadata][tenant]" | ||
| dictionary_path => "/etc/conf.d/logstash/tenants.yml" | ||
| } | ||
| } | ||
| output { <3> | ||
| if [@metadata][tenant] == "tenant01" { | ||
| elasticsearch { | ||
| cloud_id => "<cloud id>" | ||
| api_key => "<api key>" | ||
| } | ||
| } else if [@metadata][tenant] == "tenant02" { | ||
| elasticsearch { | ||
| cloud_id => "<cloud id>" | ||
| api_key => "<api key>" | ||
| } | ||
| } | ||
| } | ||
| ----- | ||
|
|
||
| <1> Use `filter-elastic_agent` as the first filter in your pipeline | ||
| <2> You can use additional filters as long as they follow `filter-elastic_agent` | ||
| <3> Sample config to output data to multiple destinations |
Oops, something went wrong.