Bump VPA to upstream

vertical-pod-autoscaler/FAQ.md

@@ -3,6 +3,7 @@
 ## Contents
 
 - [VPA restarts my pods but does not modify CPU or memory settings. Why?](#vpa-restarts-my-pods-but-does-not-modify-CPU-or-memory-settings)
+- [How can I apply VPA to my Custom Resource?](#how-can-i-apply-vpa-to-my-custom-resource)
 - [How can I use Prometheus as a history provider for the VPA recommender?](#how-can-i-use-prometheus-as-a-history-provider-for-the-vpa-recommender)
 - [I get recommendations for my single pod replicaSet, but they are not applied. Why?](#i-get-recommendations-for-my-single-pod-replicaset-but-they-are-not-applied)
 - [What are the parameters to VPA recommender?](#what-are-the-parameters-to-vpa-recommender)
@@ -107,6 +108,18 @@ is serving.
 
 Note: the commands will differ if you deploy VPA in a different namespace.
 
+### How can I apply VPA to my Custom Resource?
+
+The VPA can scale not only the built-in resources like Deployment or StatefulSet, but also Custom Resources which manage
+Pods. Just like the Horizontal Pod Autoscaler, the VPA requires that the Custom Resource implements the
+[`/scale` subresource](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/#scale-subresource)
+with the optional field `labelSelector`,
+which corresponds to `.scale.status.selector`. VPA doesn't use the `/scale` subresource for the actual scaling, but uses
+this label selector to identify the Pods managed by a Custom Resource. As VPA relies on Pod eviction to apply new
+resource recommendations, this ensures that all Pods with a matching VPA object are managed by a controller that will
+recreate them after eviction. Furthermore, it avoids misconfigurations that happened in the past when label selectors
+were specified manually.
+
 ### How can I use Prometheus as a history provider for the VPA recommender
 
 Configure your Prometheus to get metrics from cadvisor. Make sure that the metrics from the cadvisor have the label `job=kubernetes-cadvisor`

vertical-pod-autoscaler/OWNERS

@@ -1,11 +1,14 @@
 approvers:
+- kwiesmueller
 - kgolab
 - jbartosik
 - krzysied
 reviewers:
+- kwiesmueller
 - kgolab
 - jbartosik
 - krzysied
+- voelzmo
 emeritus_approvers:
 - schylek # 2022-09-30
 labels:

vertical-pod-autoscaler/README.md

@@ -15,12 +15,14 @@
   - [Troubleshooting](#troubleshooting)
   - [Components of VPA](#components-of-vpa)
   - [Tear down](#tear-down)
+- [Limits control](#limits-control)
 - [Examples](#examples)
   - [Keeping limit proportional to request](#keeping-limit-proportional-to-request)
   - [Capping to Limit Range](#capping-to-limit-range)
   - [Resource Policy Overriding Limit Range](#resource-policy-overriding-limit-range)
   - [Starting multiple recommenders](#starting-multiple-recommenders)
   - [Using CPU management with static policy](#using-cpu-management-with-static-policy)
+  - [Controlling eviction behavior based on scaling direction and resource](#controlling-eviction-behavior-based-on-scaling-direction-and-resource)
 - [Known limitations](#known-limitations)
 - [Related links](#related-links)
 
@@ -48,12 +50,14 @@ procedure described below.
 
 # Installation
 
-The current default version is Vertical Pod Autoscaler 0.13.0
+The current default version is Vertical Pod Autoscaler 0.14.0
 
 ### Compatibility
 
 | VPA version     | Kubernetes version |
 |-----------------|--------------------|
+| 1.0             | 1.25+              |
+| 0.14            | 1.25+              |
 | 0.13            | 1.25+              |
 | 0.12            | 1.25+              |
 | 0.11            | 1.22 - 1.24        |
@@ -63,11 +67,36 @@ The current default version is Vertical Pod Autoscaler 0.13.0
 | 0.4 to 0.7      | 1.11+              |
 | 0.3.X and lower | 1.7+               |
 
+### Notice on CRD update (>=1.0.0)
+**NOTE:** In version 1.0.0, we have updated the CRD definition and added RBAC for the
+status resource. If you are upgrading from version (<=0.14.0), you must update the CRD
+definition and RBAC.
+```shell
+kubectl apply -f https://raw.githubusercontent.com/kubernetes/autoscaler/vpa-release-1.0/vertical-pod-autoscaler/deploy/vpa-v1-crd-gen.yaml
+kubectl apply -f https://raw.githubusercontent.com/kubernetes/autoscaler/vpa-release-1.0/vertical-pod-autoscaler/deploy/vpa-rbac.yaml
+```
+Another method is to re-execute the ./hack/vpa-process-yamls.sh script.
+```shell
+git clone https://github.com/kubernetes/autoscaler.git
+cd autoscaler/vertical-pod-autoscaler
+git checkout origin/vpa-release-1.0
+REGISTRY=registry.k8s.io/autoscaling TAG=1.0.0 ./hack/vpa-process-yamls.sh apply
+```
+
+If you need to roll back to version (<=0.14.0), please check out the release for your
+rollback version and execute ./hack/vpa-process-yamls.sh. For example, to rollback to 0.14.0:
+```shell
+git checkout origin/vpa-release-0.14
+REGISTRY=registry.k8s.io/autoscaling TAG=0.14.0 ./hack/vpa-process-yamls.sh apply
+kubectl delete clusterrole system:vpa-status-actor
+kubectl delete clusterrolebinding system:vpa-status-actor
+```
+
 ### Notice on deprecation of v1beta2 version (>=0.13.0)
 **NOTE:** In 0.13.0 we deprecate `autoscaling.k8s.io/v1beta2` API. We plan to
 remove this API version. While for now you can continue to use `v1beta2` API we
 recommend using `autoscaling.k8s.io/v1` instead. `v1` and `v1beta2` APIs are
-almost identical (`v1` API has some fields which are not present in `v1beta2)
+almost identical (`v1` API has some fields which are not present in `v1beta2`)
 so simply changing which API version you're calling should be enough in almost
 all cases.
 
@@ -258,14 +287,16 @@ kubectl delete clusterrolebinding myname-cluster-admin-binding
 # Limits control
 
 When setting limits VPA will conform to
-[resource policies](https://github.com/kubernetes/autoscaler/blob/master/vertical-pod-autoscaler/pkg/apis/autoscaling.k8s.io/v1/types.go#L82).
+[resource policies](https://github.com/kubernetes/autoscaler/blob/master/vertical-pod-autoscaler/pkg/apis/autoscaling.k8s.io/v1/types.go#L100).
 It will maintain limit to request ratio specified for all containers.
 
 VPA will try to cap recommendations between min and max of
 [limit ranges](https://kubernetes.io/docs/concepts/policy/limit-range/). If limit range conflicts
-and VPA resource policy conflict, VPA will follow VPA policy (and set values outside the limit
+with VPA resource policy, VPA will follow VPA policy (and set values outside the limit
 range).
 
+To disable getting VPA recommendations for an individual container, set `mode` to `"Off"` in `containerPolicies`.
+
 ## Examples
 
 ### Keeping limit proportional to request
@@ -300,14 +331,14 @@ For example you could have 3 profiles: [frugal](deploy/recommender-deployment-lo
 use different TargetCPUPercentile (50, 90 and 95) to calculate their recommendations.
 
 Please note the usage of the following arguments to override default names and percentiles:
-- --name=performance
+- --recommender-name=performance
 - --target-cpu-percentile=0.95
 
 You can then choose which recommender to use by setting `recommenders` inside the `VerticalPodAutoscaler` spec.
 
 
 ### Custom memory bump-up after OOMKill
-After an OOMKill event was observed, VPA increases the memory recommendation based on the observed memory usage in the event according to this formula: `recommendation = memory-usage-in-oomkill-event + max(oom-min-bump-up-bytes, memory-usage-in-oomkill-event * oom-bump-up-ratio)`. 
+After an OOMKill event was observed, VPA increases the memory recommendation based on the observed memory usage in the event according to this formula: `recommendation = memory-usage-in-oomkill-event + max(oom-min-bump-up-bytes, memory-usage-in-oomkill-event * oom-bump-up-ratio)`.
 You can configure the minimum bump-up as well as the multiplier by specifying startup arguments for the recommender:
 `oom-bump-up-ratio` specifies the memory bump up ratio when OOM occurred, default is `1.2`. This means, memory will be increased by 20% after an OOMKill event.
 `oom-min-bump-up-bytes` specifies minimal increase of memory after observing OOM. Defaults to `100 * 1024 * 1024` (=100MiB)
@@ -324,14 +355,25 @@ Usage in recommender deployment
 ### Using CPU management with static policy
 
 If you are using the [CPU management with static policy](https://kubernetes.io/docs/tasks/administer-cluster/cpu-management-policies/#static-policy) for some containers,
-you probably want the CPU recommendation to be an integer. A dedicated recommendation pre-processor can perform a round up on the CPU recommendation. Recommendation capping still applies after the round up.   
-To activate this feature, pass the flag `--cpu-integer-post-processor-enabled` when you start the recommender. 
+you probably want the CPU recommendation to be an integer. A dedicated recommendation pre-processor can perform a round up on the CPU recommendation. Recommendation capping still applies after the round up.
+To activate this feature, pass the flag `--cpu-integer-post-processor-enabled` when you start the recommender.
 The pre-processor only acts on containers having a specific configuration. This configuration consists in an annotation on your VPA object for each impacted container.
 The annotation format is the following:
 ```
 vpa-post-processor.kubernetes.io/{containerName}_integerCPU=true
 ```
 
+### Controlling eviction behavior based on scaling direction and resource
+ To limit disruptions caused by evictions, you can put additional constraints on the Updater's eviction behavior by specifying `.updatePolicy.EvictionRequirements` in the VPA spec. An `EvictionRequirement` contains a resource and a `ChangeRequirement`, which is evaluated by comparing a new recommendation against the currently set resources for a container.
+ Here is an example configuration which allows evictions only when CPU or memory get scaled up, but not when they both are scaled down
+ ```
+ updatePolicy:
+   evictionRequirements:
+     - resources: ["cpu", "memory"]
+       changeRequirement: TargetHigherThanRequests
+ ```
+ Note that this doesn't prevent scaling down entirely, as Pods may get recreated for different reasons, resulting in a new recommendation being applied. See [the original AEP](https://github.com/kubernetes/autoscaler/tree/master/vertical-pod-autoscaler/enhancements/4831-control-eviction-behavior) for more context and usage information.
+
 # Known limitations
 
 * Whenever VPA updates the pod resources, the pod is recreated, which causes all
@@ -341,8 +383,7 @@ vpa-post-processor.kubernetes.io/{containerName}_integerCPU=true
   (when configured in `Auto` and `Recreate` modes) will be successfully
   recreated. This can be partly
   addressed by using VPA together with [Cluster Autoscaler](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#basics).
-* VPA does not evict pods which are not run under a controller. For such pods
-  `Auto` mode is currently equivalent to `Initial`.
+* VPA does not update resources of pods which are not run under a controller.
 * Vertical Pod Autoscaler **should not be used with the [Horizontal Pod Autoscaler](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#support-for-resource-metrics) (HPA) on CPU or memory** at this moment.
   However, you can use VPA with [HPA on custom and external metrics](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#scaling-on-custom-metrics).
 * The VPA admission controller is an admission webhook. If you add other admission webhooks

vertical-pod-autoscaler/RELEASE.md

@@ -23,76 +23,91 @@ We use the issue to communicate what is state of the release.
 ## Update VPA version const
 
 1. [ ] Wait for all VPA changes that will be in the release to merge.
-2. [ ] Wait for [the end to end tests](https://k8s-testgrid.appspot.com/sig-autoscaling-vpa) to run with all VPA changes
+2. [ ] Wait for [the end to end tests](https://testgrid.k8s.io/sig-autoscaling-vpa) to run with all VPA changes
    included.
+   To see what code was actually tested, look for `===== last commit =====`
+   entries in the full `build-log.txt` of a given test run.
 3. [ ] Make sure the end to end VPA tests are green.
 
 ### New minor release
 
 1. [ ] Change the version in
     [common/version-go](https://github.com/kubernetes/autoscaler/blob/master/vertical-pod-autoscaler/common/version.go)
-    to `0.${next-minor}.0`,
+    to `1.${next-minor}.0`,
 2. [ ] Commit and merge the change,
 3. [ ] Go to the merged change,
-4. [ ] [Create a new branch](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-and-deleting-branches-within-your-repository) named `vpa-release-0.${next-minor}` from the
+4. [ ] [Create a new branch](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-and-deleting-branches-within-your-repository) named `vpa-release-1.${next-minor}` from the
     merged change.
 
 ### New patch release
 
 1.  [ ] Bump the patch version number in VerticalPodAutoscalerVersion constant in
     [common/version.go](https://github.com/kubernetes/autoscaler/blob/master/vertical-pod-autoscaler/common/version.go).
-    Create a commit and merge by making a PR to the `vpa-release-0.${minor}` branch.
+    Create a commit and merge by making a PR to the `vpa-release-1.${minor}` branch.
 
 ## Build and stage images
 
+Create a fresh clone of the repo and switch to the `vpa-release-1.${minor}`
+branch. This makes sure you have no local changes while building the images.
+
+For example:
 ```sh
-for component in recommender updater admission-controller ; do REGISTRY=gcr.io/k8s-staging-autoscaling TAG=[*vpa-version*] make release --directory=pkg/${component}; done
+git clone [email protected]:kubernetes/autoscaler.git
+git switch vpa-release-1.0
+```
+
+Once in the freshly cloned repo, build and stage the images.
+
+```sh
+cd vertical-pod-autoscaler/
+for component in recommender updater admission-controller ; do TAG=`grep 'const VerticalPodAutoscalerVersion = ' common/version.go | cut -d '"' -f 2` REGISTRY=gcr.io/k8s-staging-autoscaling make release --directory=pkg/${component}; done
 ```
 
 ## Test the release
 
 1.  [ ] Create a Kubernetes cluster. If you're using GKE you can use the following command:
 
     ```shell
-    $ gcloud container clusters create e2e-test --machine-type=n1-standard-2 --image-type=COS_CONTAINERD --num-nodes=3
+    gcloud container clusters create e2e-test --machine-type=n1-standard-2 --image-type=COS_CONTAINERD --num-nodes=3
     ```
 
 1. [ ]  Create clusterrole. If you're using GKE you can use the following command:
 
     ```shell
-    $ kubectl create clusterrolebinding my-cluster-admin-binding --clusterrole=cluster-admin --user=`gcloud config get-value account`
+    kubectl create clusterrolebinding my-cluster-admin-binding --clusterrole=cluster-admin --user=`gcloud config get-value account`
     ```
 
 1.  [ ] Deploy VPA:
     ```shell
-    $ REGISTRY=gcr.io/k8s-staging-autoscaling TAG=[*vpa-version*] ./hack/vpa-up.sh
+    REGISTRY=gcr.io/k8s-staging-autoscaling TAG=`grep 'const VerticalPodAutoscalerVersion = ' common/version.go | cut -d '"' -f 2` ./hack/vpa-up.sh
     ```
 
 1.  [ ] [Run](https://github.com/kubernetes/autoscaler/blob/master/vertical-pod-autoscaler/hack/run-e2e-tests.sh)
     the `full-vpa` test suite:
 
     ```shell
-    $ ./hack/run-e2e-tests.sh full-vpa
+    ./hack/run-e2e-tests.sh full-vpa
     ```
 
 ## Promote image
 
 To promote image from staging repo send out PR updating
-[autoscaling images mapping](https://github.com/kubernetes/k8s.io/blob/master/k8s.gcr.io/images/k8s-staging-autoscaling/images.yaml)
+[autoscaling images mapping](https://github.com/kubernetes/k8s.io/blob/master/registry.k8s.io/images/k8s-staging-autoscaling/images.yaml)
 ([example](https://github.com/kubernetes/k8s.io/pull/1318)).
 
+NOTE: Please use the [add-version.sh
+script](https://github.com/kubernetes/k8s.io/blob/main/registry.k8s.io/images/k8s-staging-autoscaling/add-version.sh)
+to prepare the changes automatically.
+
 When PR merges the promoter will run automatically and upload the image from
 staging repo to final repo. The post submit job status can be tracked on
 [testgrid](https://testgrid.k8s.io/sig-k8s-infra-k8sio#post-k8sio-image-promo).
 To verify if the promoter finished its job one can use gcloud. E.g.:
 
 ```sh
-$ gcloud container images describe us.gcr.io/k8s-artifacts-prod/autoscaling/vpa-recommender:[*vpa-version*]
+gcloud container images describe registry.k8s.io/autoscaling/vpa-recommender:[*vpa-version*]
 ```
 
-You can also take a look at the images in the
-[k8s-artifacts repo](https://us.gcr.io/k8s-artifacts-prod/autoscaling/).
-
 ## Finalize release
 
 NOTE: We currently use two tags for releases:
@@ -112,7 +127,7 @@ sure nothing we care about will break if we do.
     ```sh
     sed -i -s "s|[0-9]\+\.[0-9]\+\.[0-9]\+|[*vpa-version*]|" ./deploy/*-deployment*.yaml ./hack/vpa-process-yaml.sh
     ```
-   Merge this change into branch vpa-release-0.X and optionally into master if 0.X is the latest minor release (example
+   Merge this change into branch vpa-release-1.{$minor} and optionally into master if 1.{$minor} is the latest minor release (example
    PR: [#5460](https://github.com/kubernetes/autoscaler/pull/5460)).
 
 1.  [ ] Tag the commit with version const change
@@ -131,7 +146,7 @@ sure nothing we care about will break if we do.
 
 1.  [ ] To create and publish a github release from pushed tag go to
     https://github.com/kubernetes/autoscaler/releases/tag/vertical-pod-autoscaler-[*vpa-version*],
-    press `Edit release`, complete release title and release notes, tick the
+    press `Create release from tag`, complete release title and release notes, tick the
     `This is a pre-release` box and press `Publish release`.
 
 ## Permissions
@@ -156,4 +171,4 @@ sure nothing we care about will break if we do.
 
     A member of the
     [autoscaler-admins](https://github.com/orgs/kubernetes/teams/autoscaler-admins)
-    can add you to add you as a collaborator.
+    can add you to add you as a collaborator.

vertical-pod-autoscaler/common/version.go

@@ -17,4 +17,4 @@ limitations under the License.
 package common
 
 // VerticalPodAutoscalerVersion is the version of VPA.
-const VerticalPodAutoscalerVersion = "0.13.0"
+const VerticalPodAutoscalerVersion = "1.0.0"

vertical-pod-autoscaler/deploy/admission-controller-deployment.yaml

@@ -20,7 +20,7 @@ spec:
         runAsUser: 65534 # nobody
       containers:
         - name: admission-controller
-          image: registry.k8s.io/autoscaling/vpa-admission-controller:0.13.0
+          image: registry.k8s.io/autoscaling/vpa-admission-controller:1.0.0
           imagePullPolicy: Always
           env:
             - name: NAMESPACE

vertical-pod-autoscaler/deploy/recommender-deployment-high.yaml

@@ -26,10 +26,10 @@ spec:
         runAsUser: 65534 # nobody
       containers:
       - name: recommender
-        image: registry.k8s.io/autoscaling/vpa-recommender:0.13.0
+        image: registry.k8s.io/autoscaling/vpa-recommender:1.0.0
         imagePullPolicy: Always
         args:
-          - --name=performance
+          - --recommender-name=performance
           - --target-cpu-percentile=0.95
         resources:
           limits:

vertical-pod-autoscaler/deploy/recommender-deployment-low.yaml

@@ -26,10 +26,10 @@ spec:
         runAsUser: 65534 # nobody
       containers:
       - name: recommender
-        image: registry.k8s.io/autoscaling/vpa-recommender:0.13.0
+        image: registry.k8s.io/autoscaling/vpa-recommender:1.0.0
         imagePullPolicy: Always
         args:
-          - --name=frugal
+          - --recommender-name=frugal
           - --target-cpu-percentile=0.50
         resources:
           limits:

vertical-pod-autoscaler/deploy/recommender-deployment.yaml

@@ -20,7 +20,7 @@ spec:
         runAsUser: 65534 # nobody
       containers:
       - name: recommender
-        image: registry.k8s.io/autoscaling/vpa-recommender:0.13.0
+        image: registry.k8s.io/autoscaling/vpa-recommender:1.0.0
         imagePullPolicy: Always
         resources:
           limits:

vertical-pod-autoscaler/deploy/updater-deployment.yaml

@@ -20,7 +20,7 @@ spec:
         runAsUser: 65534 # nobody
       containers:
         - name: updater
-          image: registry.k8s.io/autoscaling/vpa-updater:0.13.0
+          image: registry.k8s.io/autoscaling/vpa-updater:1.0.0
           imagePullPolicy: Always
           env:
             - name: NAMESPACE