feat: Add jobs pod when using high availabily deployment #115
Conversation
| @@ -0,0 +1,447 @@ | |||
| {{- if eq (include "bindplane.deployment_type" .) "Deployment" }} | |||
There was a problem hiding this comment.
This deployment is based on bindplane.yaml, with anything related to statefulset removed, such as volumes or Prometheus sidecar.
All logic is identical to bindplane.yaml unless otherwise noted.
| - name: BINDPLANE_MODE | ||
| {{- if eq (include "bindplane.deployment_type" .) "StatefulSet" }} | ||
| value: all | ||
| {{- else }} | ||
| value: node | ||
| {{- end }} |
There was a problem hiding this comment.
StatefulSet is deployed as a single pod, and should always have mode all.
| {{- with .Values.jobs.resources }} | ||
| resources: | ||
| {{- toYaml . | nindent 12 }} | ||
| {{- end }} |
There was a problem hiding this comment.
Jobs specific resource configuration. In a highly scaled environment, it is possible the main pods will have low cpu and memory values, so we want the jobs pod to have its own configuration.
| app.kubernetes.io/instance: {{ .Release.Name }} | ||
| app.kubernetes.io/managed-by: {{ .Release.Service }} | ||
| spec: | ||
| replicas: 1 |
There was a problem hiding this comment.
Unlike bindplane.yaml, we always set replicas to 1 to ensure there is a single jobs pod.
During a rolling update, it is okay if multiple job pods exists during the lifetime of the pod rollout.
| labels: | ||
| app.kubernetes.io/name: {{ include "bindplane.name" . }} | ||
| app.kubernetes.io/stack: bindplane | ||
| app.kubernetes.io/component: jobs |
There was a problem hiding this comment.
The component label is jobs instead of server. The clusterIP service will not match this pod due to the labels.
jsirianni
requested review from
Mrod1598,
dpaasman00 and
antonblock
and removed request for
Mrod1598 and
dpaasman00
Description of Changes
Copied
bindplane.yamltobindplane-jobs.yaml. The newjobsdeployment will deploy when the deployment type is "Deployment" (High availability).Currently, jobs has the similar configuration as the main bindplane deployment. Notable changes:
BINDPLANE_MODEisalljobsTODO
I think in the future we can refactor both deployments so they share more configuration. Specifically, environment variables. I would like to get more robust testing in place before making such a large change.
Testing
The jobs pod should only deploy when using postgres. When postgres is configured, the deployment type will be
Deployment. When using bbolt, deployment type will beStatefulSet.Bbolt
When testing w/ bbolt, the jobs pod is not deployed and the statefulset has
mode: all.Postgres
When using Postgres, BindPlane is deployed as a multi pod Deployment along with a single pod "jobs" Deployment. The main Deployment has
mode: nodeand the single jobs pod hasmode: all. Also note that thejobspod is not present in the main bindplane clusterIP service, meaning agents and UI will not connect to it. Its only purpose is to handle periodic jobs.I created the license secret with:
kubectl create secret generic bindplane \ --from-literal=license=$BINDPLANE_LICENSEI deployed the pubsub and postgres helpers and used the values file at test/cases/pubsub.
The bindplane pods restarted a couple times, waiting for the jobs pod to handle configuring the database for initial deployment.
Once deployed, the main deployment has
mode: nodeThe jobs deployment has
mode: all.The clusterIP service does not match the job pod's ip address, meaning it will never route incoming requests to it. This is desired to keep its resource consumption low.
$ kubectl describe svc release-name-bindplane | grep Endpoints Endpoints: 10.244.0.15:3001,10.244.0.16:3001,10.244.0.18:3001The UI can be accessed by port forwarding to the service.
http://localhost:3011
Please check that the PR fulfills these requirements