Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Epic: Unify vm-builders #577

Closed
2 tasks done
sharnoff opened this issue Oct 21, 2023 · 1 comment
Closed
2 tasks done

Epic: Unify vm-builders #577

sharnoff opened this issue Oct 21, 2023 · 1 comment
Labels
c/autoscaling/neonvm Component: autoscaling: NeonVM t/Epic Issue type: Epic

Comments

@sharnoff
Copy link
Member

sharnoff commented Oct 21, 2023 •
edited
Loading

Motivation

The current vm-builder situation is messy and inflexible:

  1. We have two copies of roughly the same code
  2. Components like vector, pgbouncer, and postgres_exporter are all installed and configured in vm-builder
    • Updating the version or config requires (a) a new autoscaling release, and then (b) new compute images — whereas the control plane can immediately change configs or versions for regular pods

We also have a very hacky solution for process lifecycle management during shutdown implemented in #525 (see also #529).

DoD

  • Unified vm-builder and vm-builder-generic
  • No postgres- or Neon-specific details in vm-builder

Implementation ideas

(moved to #835)

Tasks

Tasks
Preview Give feedback
  1. vm-builder: Remove neon-specifics, add customization with -spec=<file> #600
  2. Update vm-builder to v0.19.0 and move its customization here neon#5783

Other related tasks, Epics, and links
@sharnoff sharnoff added t/Epic Issue type: Epic c/autoscaling/neonvm Component: autoscaling: NeonVM labels Oct 21, 2023
This was referenced Oct 21, 2023
Open
Merged
Merged
@sharnoff sharnoff self-assigned this Oct 30, 2023
@sharnoff sharnoff mentioned this issue Nov 1, 2023
Merged
sharnoff added a commit that referenced this issue Nov 15, 2023
@sharnoff
vm-builder: Remove neon-specifics, add customization with `-spec=<fil…
58e0271 
…e>` (#600)

First part of the "unify vm-builders" epic (#577)
This PR is the "80% gain with 20% effort" equivalent for these changes.

This customization should:

1. Make it easier to make changes to pgbouncer and/or postgres-exporter
    without requiring changes in this repo; and
2. Make it *possible* to plumb through pgbouncer customization from the
    control plane.

Other key changes:

- vm-monitor is now only used in CI (no longer released, or bundled with
  vm-builder)
- various flags removed from vm-builder: `-monitor`, `-enable-monitor`,
  `-enable-file-cache`, `-cgroup-uid`
@sharnoff sharnoff removed their assignment Dec 18, 2023
@sharnoff sharnoff mentioned this issue Jan 8, 2024
Open
@sharnoff sharnoff mentioned this issue Feb 27, 2024
Open
@sharnoff sharnoff changed the title Epic: Unify vm-builders, allow running multiple "containers" Epic: Unify vm-builders Feb 27, 2024
@sharnoff
Copy link
Member Author

Moved the "containers" part to a separate issue in #835; closing this because the first half has been completed for a while.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
c/autoscaling/neonvm Component: autoscaling: NeonVM t/Epic Issue type: Epic
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@sharnoff