OCI Logging Analytics Fluentd output plugin collects event logs, buffer into local file system, periodically creates payload and uploads it to OCI Logging Analytics.
Refer Prerequisites
| fluent-plugin-oci-logging-analytics | fluentd | ruby | rubyzip | oci | prometheus-client | yajl-ruby |
|---|---|---|---|---|---|---|
| >= 2.0.0 | >= 0.14.10, < 2 | >= 2.6 | ~> 2.3.2 | ~> 2.16 | ~> 2.1.0 | ~>1.4.3 |
Add this line to your application's Gemfile:
gem 'fluent-plugin-oci-logging-analytics'
And then execute:
$ bundle
Or install it manually as:
$ gem install fluent-plugin-oci-logging-analytics --no-document
# If you need to install specific version, use -v option
- Buffer configuration parameters
- Note* - Buffer type 'file' is recommended but no longer mandatory. Customer can configure other available options.
- In case of a fast input plugin and slow output plugin, buffer will keep on increase and file based buffer with default 50GB (can configure for higher value) can handle the output plugin delay.
We may not have huge memory configured for memory based plugin and will result in data loss.
- In case of back-end service not available (5XX exceptions), output plugin will keep retrying with the existing chunk and meanwhile, new chunks will be keep on getting scheduled.
Each chunk being 2MB size, and with chunk interval 30 sec, in case of a 30 mins outage (can be more in unforeseen cases), we need a minimum of 120MB memory allocated for memory buffer.
- In any container based logging analytics, as the logs are not saved in the containers and they are completely lost in case not consumed, file based memory provides a persistent buffer implementation.
- For container based deployments, while creating the fluentd config file, we need to consider all these edge cases and proper sizing to come up with the memory size.
- As the data loss is very critical and not all customers are aware of these cases, we prevented memory buffer. Having said that, we can go ahead and remove that limitation and let the informed customers like you can do the proper sizing and decide which option is helpful for them.
The incoming log events must be in a specific format so that the Fluentd plugin provided by Oracle can process the log data, chunk them, and transfer them to OCI Logging Analytics.
Use filter plugin (record_transformer) to transform the input log events to add OCI Logging Analytics specific fields/metadata.
-
Example configuration that can be used for monitoring syslog log
-
Example configuration that can be used for monitoring apache error log
-
Example configuration that can be used for monitoring kafka log
Refer Viewing the Logs in Logging Analytics
The plugin emits following metrics in Prometheus format, which provides stats/insights about the data being collected and processed by the plugin. Refer monitoring-prometheus for details on how to expose these and other various Fluentd metrics to Prometheus (If the requirement is to collect and monitor core Fluentd and this plugin metrics alone using Prometheus then Step1 and Step2 from the referred document can be skipped).
For prometheus metrics to work properly, please add 'tag' and 'worker_id' (in case of multi worker configuration) to the filter block.
tag ${tag}
worker_id ${ENV['SERVERENGINE_WORKER_ID']}
Metric Name: oci_la_fluentd_output_plugin_records_received
labels: [:worker_id,:tag,:oci_la_log_group_id,:oci_la_log_source_name,:oci_la_log_set]
Description: Number of records received by the OCI Logging Analytics Fluentd output plugin.
Type : Gauge
Metric Name: oci_la_fluentd_output_plugin_records_valid
labels: [:worker_id,:tag,:oci_la_log_group_id,:oci_la_log_source_name,:oci_la_log_set]
Description: Number of valid records received by the OCI Logging Analytics Fluentd output plugin.
Type : Gauge
Metric Name: oci_la_fluentd_output_plugin_records_invalid
labels: [:worker_id,:tag,:oci_la_log_group_id,:oci_la_log_source_name,:oci_la_log_set,:reason]
Description: Number of invalid records received by the OCI Logging Analytics Fluentd output plugin.
Type : Gauge
Metric Name: oci_la_fluentd_output_plugin_records_post_error
labels: [:worker_id,:tag,:oci_la_log_group_id,:oci_la_log_source_name,:oci_la_log_set,:error_code, :reason]
Description: Number of records failed posting to OCI Logging Analytics by the Fluentd output plugin.
Type : Gauge
Metric Name: oci_la_fluentd_output_plugin_records_post_success
labels: [:worker_id,:tag,:oci_la_log_group_id,:oci_la_log_source_name,:oci_la_log_set]
Description: Number of records posted by the OCI Logging Analytics Fluentd output plugin.
Type : Gauge
Metric Name: oci_la_fluentd_output_plugin_chunk_time_to_receive
labels: [:worker_id,:tag]
Description: Average time taken by Fluentd to deliver the collected records from Input plugin to OCI Logging Analytics output plugin.
Type : Histogram
Metric Name: oci_la_fluentd_output_plugin_chunk_time_to_post
labels: [:oci_la_log_group_id]
Description: Average time taken for posting the received records to OCI Logging Analytics by the Fluentd output plugin.
Type : Histogram
See FAQ.
See CHANGELOG.
Copyright (c) 2021, 2022 Oracle and/or its affiliates.
See LICENSE for more details.