Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(ingest/kafka-connect): update connection test url, handle api failures #12082

Merged
merged 3 commits into from
Dec 13, 2024
Merged

fix(ingest/kafka-connect): update connection test url, handle api failures #12082

merged 3 commits into from
Dec 13, 2024

Conversation

mayurinehate
Copy link
Collaborator

@mayurinehate mayurinehate commented Dec 10, 2024 •
edited
Loading

When using confluent cloud,

  • GET does work on <connect-uri>/ but works on <connect-uri>/connectors endpoint when testing auth. We achieve the intent of testing connection even if we use any of these urls.
  • GET on /connectors/{connector-name}/topic is not supported, so lineage can not be extracted. The PR adds exception handling for failure in accessing endpoint and attempts to extract rest of information.
  • Refactors around reporting warnings for structured reporting

Checklist

  • The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
  • Links to related issues (if applicable)
  • Tests for the changes have been added/updated (if applicable)
  • Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
  • For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

@github-actions github-actions bot added the ingestion PR or Issue related to the ingestion of metadata label Dec 10, 2024
@datahub-cyborg datahub-cyborg bot added the needs-review Label for PRs that need review from a maintainer. label Dec 10, 2024
@mayurinehate
fix(ingest/kafka-connect): update connection test url
86cd304 
When using confluent cloud, GET does work on root but works on
/connectors endpoint
@mayurinehate mayurinehate force-pushed the kafka_connect_test_fix branch from c01cb59 to 86cd304 Compare December 11, 2024 06:18
@mayurinehate mayurinehate marked this pull request as draft December 11, 2024 12:52
@mayurinehate mayurinehate changed the title fix(ingest/kafka-connect): update connection test url fix(ingest/kafka-connect): update connection test url, handle api failures Dec 12, 2024
@mayurinehate mayurinehate marked this pull request as ready for review December 12, 2024 13:39
sgomezvillamor
sgomezvillamor approved these changes Dec 12, 2024
View reviewed changes
Copy link
Contributor

@sgomezvillamor sgomezvillamor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM
minor suggestions

@datahub-cyborg datahub-cyborg bot added pending-submitter-merge and removed needs-review Label for PRs that need review from a maintainer. labels Dec 12, 2024
@mayurinehate mayurinehate merged commit d5e0513 into datahub-project:master Dec 13, 2024
74 of 75 checks passed
sleeperdeep pushed a commit to sleeperdeep/datahub that referenced this pull request Dec 17, 2024
@mayurinehate @sleeperdeep
fix(ingest/kafka-connect): update connection test url, handle api fai…
7b4ba97 
…lures (datahub-project#12082)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sgomezvillamor sgomezvillamor sgomezvillamor approved these changes

Assignees
No one assigned
Labels
ingestion PR or Issue related to the ingestion of metadata pending-submitter-merge
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@mayurinehate @sgomezvillamor