Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for migrations within compute_ctl #6085

Merged
merged 22 commits into from
Jan 22, 2024
Merged

Conversation

save-buffer
Copy link
Contributor

@save-buffer save-buffer commented Dec 8, 2023

Problem

It's easy bungle stuff with regards to permissions or other stuff. Fixing it is hard because we don't have a way to run a query on everyone's computes exactly once. Any migrations we currently do makes it so that we have to maek sure everything is ok to be run many times.

Summary of changes

Executes migrations on startup and persists the migrations inside of a table with a single row.

Checklist before requesting a review

  • I have performed a self-review of my code.
  • If it is a core feature, I have added thorough tests.
  • Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
  • If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

  • Do not forget to reformat commit message to not include the above checklist

@save-buffer save-buffer requested review from a team as code owners December 8, 2023 21:37
@save-buffer save-buffer requested review from NanoBjorn, knizhnik and vadim2404 and removed request for a team December 8, 2023 21:37
Copy link

github-actions bot commented Dec 8, 2023

2268 tests run: 2184 passed, 0 failed, 84 skipped (full report)


Flaky tests (3)

Postgres 15

  • test_secondary_mode_eviction: debug
  • test_tenant_delete_races_timeline_creation: debug

Postgres 14

  • test_logical_replication: release

Code coverage (full report)

  • functions: 54.5% (10592 of 19429 functions)
  • lines: 81.5% (60576 of 74351 lines)

The comment gets automatically updated with the latest test results
d2e0d23 at 2024-01-18T22:13:04.545Z :recycle:

@save-buffer save-buffer force-pushed the sasha_migrations branch 2 times, most recently from eff9d16 to 458c554 Compare December 12, 2023 18:34
@NanoBjorn NanoBjorn removed their request for review December 19, 2023 12:04
@vadim2404
Copy link
Contributor

@kelvich , you wanted to review this PR

@vadim2404 vadim2404 requested a review from kelvich December 19, 2023 14:36
Copy link
Contributor

@kelvich kelvich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good generally. I'd change migration tracking to be a bit more robust

compute_tools/src/spec.rs Show resolved Hide resolved
compute_tools/src/spec.rs Show resolved Hide resolved
compute_tools/src/compute.rs Show resolved Hide resolved
compute_tools/src/spec.rs Show resolved Hide resolved
compute_tools/src/spec.rs Show resolved Hide resolved
compute_tools/src/spec.rs Outdated Show resolved Hide resolved
@save-buffer save-buffer force-pushed the sasha_migrations branch 4 times, most recently from 8f9bbc2 to 120edb5 Compare January 9, 2024 23:12
@save-buffer save-buffer requested a review from kelvich January 10, 2024 16:38
Copy link
Contributor

@kelvich kelvich left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that is in a good state right now.

wonder if there is an easy way to feature-flag this and gradually roll it out. or at least region by region.

@kelvich
Copy link
Contributor

kelvich commented Jan 11, 2024

cc @ololobus to be aware = )

@kelvich
Copy link
Contributor

kelvich commented Jan 11, 2024

also tests are broken

@ololobus
Copy link
Member

wonder if there is an easy way to feature-flag this and gradually roll it out. or at least region by region.

See for compute feature flags

I'm going to get use of them in #6008

@save-buffer save-buffer requested a review from a team as a code owner January 18, 2024 19:17
@save-buffer save-buffer requested review from VladLazar and removed request for a team January 18, 2024 19:17
@save-buffer
Copy link
Contributor Author

Added support for these feature flags in https://github.com/neondatabase/cloud/pull/9450, will merge this PR once that one finishes

@save-buffer save-buffer merged commit 71f495c into main Jan 22, 2024
46 checks passed
@save-buffer save-buffer deleted the sasha_migrations branch January 22, 2024 22:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants