Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix primary key constraint #66

Merged
merged 4 commits into from
Jul 11, 2024
Merged

Fix primary key constraint #66

merged 4 commits into from
Jul 11, 2024

Conversation

ryannedolan
Copy link
Collaborator

@ryannedolan ryannedolan commented Jun 21, 2024

Summary

Bug fix: Drop PRIMARY KEY constraint from KEY fields. Instead, support new PRIMARY_KEY "magic field" for the purpose.

Drop MySQL from integration tests, since the images involved do not support ARM.

Details

Flink SQL supports a PRIMARY KEY constraint, but not all KEYs can satisfy the constraint. In particular, keys that are nullable will cause problems.

In addition, not all connectors support primary keys -- even if they do support keys. For example, the kafka connector does not seem to support primary keys, whereas the upsert-kafka connector requires primary keys.

To workaround this issue, we drop the KEY "magic field" and instead introduce a PRIMARY_KEY "magic field". This enables adapters to use KEY or PRIMARY_KEY depending on the required semantics.

Testing

The constraint was manually removed from existing pipelines.

@ryannedolan ryannedolan force-pushed the null-keys branch 3 times, most recently from 14ed43a to c824193 Compare July 9, 2024 17:45
@ryannedolan ryannedolan changed the title Fix primary key constraint for nullable keys Drop primary key constraint Jul 10, 2024
@ryannedolan ryannedolan changed the title Drop primary key constraint Fix primary key constraint Jul 10, 2024
@ryannedolan ryannedolan enabled auto-merge (squash) July 10, 2024 16:18
@ryannedolan ryannedolan requested review from ehoner and hshukla July 10, 2024 16:19
@ryannedolan ryannedolan merged commit 92f1864 into main Jul 11, 2024
1 check passed
@ryannedolan ryannedolan deleted the null-keys branch July 11, 2024 19:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants