Filtering doesn't work with JSONed logs after migrating to OTEL collector

[prysmakou-sa]

After migrating to OTEL collector our JSONed logs aren't processed by filters.

Steps to reproduce:

Add a filter (eg to wipe out emails):

sumologic:
  metrics:
    enabled: false
  setup:
    force: true
  traces:
    enabled: false
  logs:
    priorityClassName: ${priorityClassName}
    statefulset:
      priorityClassName: ${statefulsetPriorityClassName}
    container:
      otelcol:
        extraProcessors:
          - transform/mask-email:
              log_statements:
                - context: log
                  statements:
                    - replace_pattern(body, "([a-zA-Z0-9_.+i\\-]+@)([a-zA-Z0-9\\-]+)(\\.[a-zA-Z0-9.]+)", "xxxxxx")

from your cluster run:
kubectl run -it logtest --image=node:18-alpine --restart=Never --rm -- node -e "console.error(JSON.stringify({level:'error', message:'test', data:{ email: '[email protected]' }}));"

Expected result: the email is filtered out from the logs in Sumologic.
The real result: the filter is ignored.

Run a similar command for the regular/string logs (to be sure it's not an issue with the filter syntax):

kubectl run -it logtest --image=node:18-alpine --restart=Never --rm -- node -e "console.error('email: [email protected]');"

Some more details about our environment:
EKS 1.25
helm chart 3.15.0

[swiatekm]

This is fixed by #3281 and will be present in the next release (3.16).

As a workaround, you can do the following:

- set(body, Concat([body], ""))
- replace_pattern(body, "([a-zA-Z0-9_.+i\\-]+@)([a-zA-Z0-9\\-]+)(\\.[a-zA-Z0-9.]+)", "xxxxxx")
- set(body, ParseJSON(body)) where IsMatch(body, "^{")

[prysmakou-sa]

- set(body, ParseJSON(body)) where IsMatch(body, "^{")

I tested it in our environment and it works! Thank you @swiatekm-sumo !

[swiatekm]

#3281 was merged, so I'm going to close this issue for now. If the upcoming 3.16 release doesn't fix your issue @prysmakou-sa , please reopen it.