Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[INLONG-11495][Sort] Add end-to-end test for Kafka connector v1.18 #11554

Merged
merged 1 commit into from
Dec 2, 2024

Conversation

PeterZh6
Copy link
Contributor

@PeterZh6 PeterZh6 commented Nov 28, 2024

Fixes #11495
Fixes #11553

Motivation

To verify the correctness of #11494 in a formal way.
To fix the problem of #11494, which is documented in #11553

Modifications

For #11495, added end2end test for Kafka connector by producing kafka message with kafka-clients dependency and extract the information with Kafka connector introduced by #11494. To better automate the whole process, the result is loaded to Elasticsearch7 with the existing elsasticsearch7-inlong connector. The result is then retrieved by elasticsearch client dependency and compared with the expected result.

Also, some unused dependecies are removed and the test connects to elasticsearch with co.elastic instead of high-level client.

For #11553, the package name in meta-inf of the Kafka connnector is corrected, so that the DynamicFactory class can work properly.

Verifying this change

  • This change is a trivial rework/code cleanup without any test coverage.

  • This change is already covered by existing tests, such as:

  • This change added tests and can be verified as follows:

switch to inlong/inlong-sort/sort-end-to-end-tests/sort-end-to-end-tests-v1.18 directory and run mvn clean test -Dtest=Kafka2Elasticsearch7Test
ES_end2end

Documentation

  • Does this pull request introduce a new feature? No
  • If yes, how is the feature documented? N/A
  • If a feature is not applicable for documentation, explain why? Test introduced. No need for documentation

@dockerzhang
Copy link
Contributor

@vernedeng PTAL

@vernedeng vernedeng merged commit 7be09a4 into apache:master Dec 2, 2024
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants