[INLONG-11199][Sort] Integrate Grafana Loki for connectors

[qy-liuhuo]

• Fixes [Feature][Sort] Integrate log management systems for connectors #11199

Motivation

Add the Grafana Loki as a log management system to manage logs efficiently and centrally

Modifications

• Added opentelemetry-collector, Grafna and Loki containers in the docker-compose
• Added the loki.yaml file to configure the Loki
• Optimized the OpenTelemetryLogger：
  • Added service.namespace and host.name configuration items
  • Provided a Builder to help build OpenTelemetryLogger objects more conveniently

Verifying this change

(Please pick either of the following options)

• This change is a trivial rework/code cleanup without any test coverage.

• This change is already covered by existing tests, such as:
  (please describe tests)

• This change added tests and can be verified as follows:

  (example:)

  • Added integration tests for end-to-end deployment with large payloads (10MB)
  • Extended integration test for recovery after broker failure

Documentation

After running docker-compose, you can access Grafna Loki via http://127.0.0.1:3000/explore, and then you should query the corresponding sort logs by service_name or service_namespace.

Then You can see a list of logs and view detailed information by clicking on a specific log item.

[aloyszhang]

I found there are some duplicated modifications with #11070.
Please fix this problem if both of these two pull requests are needed

[qy-liuhuo]

I found there are some duplicated modifications with #11070. Please fix this problem if both of these two pull requests are needed

Yeah，#11070 is just an example of usage， so i think we can temporarily close it（#11070）.

After this issue(#11199 ) is completed, We can raise a new issue to integrate for all connectors

[qy-liuhuo]

This new commit makes log reporting related containers optional：
When running docker-compose normally, only necessary containers will be started.
If you need to enable the opentelemetry log reporting function of the Sort module, you can add the --profile sort-report option when starting, for example, docker compose --profile sort-report up -d

To support the profiles feature, we need to update the version setting of docker-compose.yml from 2.4 to 3.0, and the docker compose version needs to be higher than 1.28.0.

[aloyszhang]

There is a problem in the workflow, please rebase the master branch to resolve it.

[qy-liuhuo]

There is a problem in the workflow, please rebase the master branch to resolve it.

ok

[aloyszhang]

Unit test failed, details refer to https://github.com/apache/inlong/actions/runs/11284951233/job/31393310968?pr=11212

[qy-liuhuo]

Unit test failed, details refer to https://github.com/apache/inlong/actions/runs/11284951233/job/31393310968?pr=11212

I got the error message: java.lang.RuntimeException: ru.yandex.clickhouse.except.ClickHouseUnknownException: ClickHouse exception, code: 1002, host: localhost, port: 32772; localhost:32772 failed to respond , it seems ooks like a problem with the ckickhouse container. Could you please restart the workflow for me?

[aloyszhang]

I got the error message: java.lang.RuntimeException: ru.yandex.clickhouse.except.ClickHouseUnknownException: ClickHouse exception, code: 1002, host: localhost, port: 32772; localhost:32772 failed to respond , it seems ooks like a problem with the ckickhouse container. Could you please restart the workflow for me?

Sure, the failed check is re-running now