AWS for Fluent Bit 2.31.3

2.31.3

This release includes:

• Fluent Bit 1.9.10
• Amazon CloudWatch Logs for Fluent Bit 1.9.1
• Amazon Kinesis Streams for Fluent Bit 1.10.1
• Amazon Kinesis Firehose for Fluent Bit 1.7.1

Compared to 2.31.2, this release adds the following fix that we are working on getting accepted upstream:

• Bug - Resolve net.keepalive SIGSEGV issue and priority event loop corruption bug aws-for-fluent-bit:542

Same as 2.31.2, this release includes the following fixes and features that we are working on getting accepted upstream:

• Bug - Resolve cloudwatch_logs duplicate tag match SIGSEGV issue introduced in 2.29.0 aws-for-fluent-bit:542
• Feature - Add kinesis_firehose and kinesis_streams support for time_key_format milliseconds with %3N option, and nanoseconds 9N and %L options fluent-bit:2831
• Feature - Support OpenSearch Serverless data ingestion via OpenSearch plugin fluent-bit:6448
• Enhancement - Transition S3 to fully synchronous file uploads to improve plugin stability fluent-bit:6573
• Bug - Mitigate Datadog output plugin issue by reverting recent PR aws-for-fluent-bit:491
• Bug - Format S3 filename with timestamp from the first log in uploaded file, rather than the time the first log was buffered by the s3 output aws-for-fluent-bit:459
• Bug - Resolve S3 logic to display log_key missing warning message if the configured log_key field is missing from log payload fluent-bit:6557
• Bug - ECS Metadata filter gracefuly handle task metadata query errors and cache metadata processing state to improve performance aws-for-fluent-bit:505
• Bug - Support Retry_Limit option in S3 plugin to set retries fluent-bit:6475
• Bug - Resolve a rare Datadog segfault that occurs when remapping tags aws-for-fluent-bit:491

---

We’ve run the new released image in our ECS load testing framework and here is the result. This testing result provides benchmarks of aws-for-fluent-bit under different input load. Learn more about the load test.

plugin	source		20 MB/s	25 MB/s	30 MB/s
kinesis_firehose	stdstream	Log Loss	✅	✅	✅
		Log Duplication	✅	✅	0%(932)
kinesis_firehose	tcp	Log Loss	✅	✅	✅
		Log Duplication	✅	✅	✅
kinesis_streams	stdstream	Log Loss	✅	✅	✅
		Log Duplication	✅	0%(9098)	0%(26069)
kinesis_streams	tcp	Log Loss	✅	✅	✅
		Log Duplication	✅	0%(1500)	0%(25373)
s3	stdstream	Log Loss	✅	12%(1862790)	23%(4223974)
		Log Duplication	✅	✅	✅
s3	tcp	Log Loss	✅	✅	15%(2783857)
		Log Duplication	✅	✅	✅

plugin	source		1 MB/s	2 MB/s	3 MB/s
cloudwatch_logs	stdstream	Log Loss	✅	✅	11%(207189)
		Log Duplication	✅	✅	✅
cloudwatch_logs	tcp	Log Loss	✅	✅	✅
		Log Duplication	✅	✅	✅