[pkg/stanza] Add regexp based tests & examples for filter operator (o…

…pen-telemetry#31698)

**Description:** <Describe what has changed.>
<!--Ex. Fixing a bug - Describe the bug and how this fixes the issue.
Ex. Adding a feature - Explain what this achieves.-->
This PR adds some additional unit tests and examples to cover the regexp
based filter-in/filter-out functionalities of the [filter
operator](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/pkg/stanza/docs/operators/filter.md).

This is an important functionality spotted during a comparison of the
`filelogreceiver`'s capabilities with those of
[Filebeat](https://github.com/elastic/beats/tree/main/filebeat#filebeat).

[Filebeat](https://github.com/elastic/beats/tree/main/filebeat#filebeat)'s
[filestream
input](https://github.com/elastic/beats/tree/main/filebeat/input/filestream)
supports this filtering with its
[include_lines](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-filestream.html#filebeat-input-filestream-include-lines)
and
[exclude_lines](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-filestream.html#filebeat-input-filestream-exclude-lines)
options.
The added tests and examples are to ensure the parity of this
capability.

**Link to tracking Issue:** <Issue number if applicable>

**Testing:** <Describe what testing was performed and which tests were
added.>
-
[FilterOutRegexp](https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/31698/files#diff-01d783f62fa2a61d9fbfe9342f0ea20629604563c550cee56b0fa0ba179556a1R50)
-
[FilterInRegexp](https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/31698/files#diff-01d783f62fa2a61d9fbfe9342f0ea20629604563c550cee56b0fa0ba179556a1R60)

**Documentation:** <Describe the documentation added.>
Added the [Filtering log messages based on
content](https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/31698/files#diff-7156c0db24790afae5276c0908dd0e02ecc7c23947607996d50008b8e4ab574dR1)
example.

PS: I'm not sure if that change technically require a changelog entry.
If so, please let me know.

Signed-off-by: ChrsMark <[email protected]>

examples/logline-filtering/README.md

@@ -0,0 +1,13 @@
+## Filtering log messages based on content
+
+Filelog receiver provides support for filtering logs based on their content. This can be achieved by using
+the [filter operator](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/pkg/stanza/docs/operators/filter.md),
+configured with matching regular expressions.
+
+With this happening at the collection point, a lot of resources at the destination backend
+can be saved since no additional processing would need to take place.
+
+A full configuration example on how to filter out logs that start with the `INFO:` pattern is
+provided in the [example config](./otel-col-config-filter-out-logs.yaml).
+A full configuration example on how to only collect logs that start with the `WARN:` pattern is provided in
+the [example config](./otel-col-config-filter-in-logs.yaml)

examples/logline-filtering/otel-col-config-filter-in-logs.yaml

@@ -0,0 +1,17 @@
+receivers:
+  filelog:
+    include: [/var/log/busybox/simple.log]
+    operators:
+      - type: filter
+        expr: 'body not matches "^WARN:"'
+
+service:
+  pipelines:
+    logs:
+      receivers: [filelog]
+      exporters: [otlp/custom]
+      processors: []
+
+exporters:
+  otlp/custom:
+    endpoint: http://0.0.0.0:4242

examples/logline-filtering/otel-col-config-filter-out-logs.yaml

@@ -0,0 +1,17 @@
+receivers:
+  filelog:
+    include: [/var/log/busybox/simple.log]
+    operators:
+      - type: filter
+        expr: 'body matches "^INFO:"'
+
+service:
+  pipelines:
+    logs:
+      receivers: [filelog]
+      exporters: [otlp/custom]
+      processors: []
+
+exporters:
+  otlp/custom:
+    endpoint: http://0.0.0.0:4242

pkg/stanza/operator/transformer/filter/filter_test.go

@@ -46,6 +46,26 @@ func TestTransformer(t *testing.T) {
 			`body.message == "test_message"`,
 			false,
 		},
+		{
+			"FilterOutRegexp",
+			&entry.Entry{
+				Body: map[string]any{
+					"message": "INFO: this is an info message",
+				},
+			},
+			`body.message matches "^INFO:"`,
+			true,
+		},
+		{
+			"FilterInRegexp",
+			&entry.Entry{
+				Body: map[string]any{
+					"message": "WARN: this is a warning message",
+				},
+			},
+			`body.message not matches "^WARN:"`,
+			false,
+		},
 		{
 			"MatchAttribute",
 			&entry.Entry{