Skip to content

Commit

Permalink
chore(test): Test pubsub and postgres in CI (#116)
Browse files Browse the repository at this point in the history
* Deploy test postgres

* add pubsub

* support pubsub dev options

* test postgres and pubsub

* rename labels test case to pubsub, implement pubsub and postgres, add to ci

* bump chart version

* set resource req and limits

* remove max parallel
  • Loading branch information
jsirianni authored Apr 16, 2024
1 parent 7e45a21 commit d380c0a
Show file tree
Hide file tree
Showing 10 changed files with 227 additions and 22 deletions.
22 changes: 20 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,13 @@ jobs:
integration:
runs-on: ubuntu-20.04
strategy:
max-parallel: 3
matrix:
case:
- "default"
- "image"
- "ingress"
- "volume"
- "pubsub"
k8s_version:
- v1.25.0
- v1.27.0
Expand All @@ -45,6 +45,12 @@ jobs:
kubernetes-version: ${{ matrix.k8s_version }}
addons: default-storageclass,storage-provisioner,ingress

- name: Deploy Postgres
run: kubectl apply -f test/helper/postgres/postgres.yaml

- name: Deploy PubSub Emulator
run: kubectl apply -f test/helper/pubsub/pubsub.yaml

- name: Wait For Ingress Pods
run: |
sleep 5
Expand All @@ -71,4 +77,16 @@ jobs:
if: always()
run: |
kubectl -n default get all
kubectl -n default logs pod/release-name-bindplane-0
kubectl -n default logs sts/release-name-bindplane || kubectl -n default logs deploy/release-name-bindplane
- name: "Debug: Get postgres status and logs"
if: always()
run: |
kubectl -n postgres get all
kubectl -n postgres logs pod/postgres-0
- name: "Debug: Get pubsub status and logs"
if: always()
run: |
kubectl -n pubsub get all
kubectl -n pubsub logs pod/pubsub-emulator-0
2 changes: 1 addition & 1 deletion charts/bindplane/Chart.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name: bindplane
description: BindPlane OP is an observability pipeline.
type: application
# The chart's version
version: 1.7.3
version: 1.7.4
# The BindPlane OP tagged release. If the user does not
# set the `image.tag` values option, this version is used.
appVersion: 1.52.0
Expand Down
4 changes: 3 additions & 1 deletion charts/bindplane/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# bindplane

![Version: 1.7.3](https://img.shields.io/badge/Version-1.7.3-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 1.52.0](https://img.shields.io/badge/AppVersion-1.52.0-informational?style=flat-square)
![Version: 1.7.4](https://img.shields.io/badge/Version-1.7.4-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 1.52.0](https://img.shields.io/badge/AppVersion-1.52.0-informational?style=flat-square)

BindPlane OP is an observability pipeline.

Expand Down Expand Up @@ -91,6 +91,8 @@ BindPlane OP is an observability pipeline.
| eventbus.kafka.topic | string | `""` | Topic to use. |
| eventbus.pubsub.credentials.secret | string | `""` | Optional Kubernetes secret which contains Google Cloud JSON service account credentials. Not required when running within Google Cloud with the Pub/Sub scope enabled. |
| eventbus.pubsub.credentials.subPath | string | `""` | Sub path for the secret which contains the Google Cloud credential JSON |
| eventbus.pubsub.endpoint | string | `""` | For testing against an emulator only. |
| eventbus.pubsub.insecure | bool | `false` | For testing against an emulator only. |
| eventbus.pubsub.projectid | string | `""` | |
| eventbus.pubsub.topic | string | `""` | |
| eventbus.type | string | `""` | |
Expand Down
8 changes: 8 additions & 0 deletions charts/bindplane/templates/bindplane.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -168,6 +168,14 @@ spec:
- name: BINDPLANE_GOOGLE_PUB_SUB_CREDENTIALS_FILE
value: /credentials.json
{{- end }}
{{- if .Values.eventbus.pubsub.endpoint }}
- name: BINDPLANE_GOOGLE_PUB_SUB_ENDPOINT
value: {{ .Values.eventbus.pubsub.endpoint }}
{{- end }}
{{- if .Values.eventbus.pubsub.insecure }}
- name: BINDPLANE_GOOGLE_PUB_SUB_INSECURE
value: "{{ .Values.eventbus.pubsub.insecure }}"
{{- end }}
{{- end }}
{{- if eq .Values.eventbus.type "kafka" }}
- name: BINDPLANE_EVENT_BUS_TYPE
Expand Down
4 changes: 4 additions & 0 deletions charts/bindplane/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,10 @@ eventbus:
secret: ""
# -- Sub path for the secret which contains the Google Cloud credential JSON
subPath: ""
# -- For testing against an emulator only.
endpoint: ""
# -- For testing against an emulator only.
insecure: false

kafka:
# -- Comma separated list of brokers to use, in the form of `host:port`.
Expand Down
17 changes: 14 additions & 3 deletions test/cases/all/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,22 @@ config:
enterprise: false
mutliAccount: false
backend:
type: bbolt
type: postgres
bbolt:
volumeSize: 2Gi
postgres:
host: postgres.postgres.svc.cluster.local
database: bindplane
username: postgres
password: password
maxConnections: 20
eventbus:
type: ""
type: 'pubsub'
pubsub:
projectid: 'my-project'
topic: 'bindplane'
insecure: true
endpoint: "pubsub-emulator.pubsub.svc.cluster.local:8681"
auth:
type: ldap
ldap:
Expand Down Expand Up @@ -50,6 +61,6 @@ trace:
otlp:
endpoint: collector.local
insecure: true
replicas: 2 # Don't do this with bolt store in production
replicas: 3
autoscaling:
enable: false
15 changes: 0 additions & 15 deletions test/cases/labels/values.yaml

This file was deleted.

43 changes: 43 additions & 0 deletions test/cases/pubsub/values.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
# Required options
config:
username: bpuser
password: bppass
secret_key: 12D8FB6E-1532-4A4C-97AF-95A430BE5E6E
sessions_secret: 4484766F-5016-4077-B8E0-0DE1D637854B
licenseUseSecret: true

extraPodLabels:
extraKey: extraValue
extraKey2: extraValue2

prometheus:
extraPodLabels:
extraKeyProm: extraValueProm
extraKeyProm2: extraValueProm2

backend:
type: postgres
postgres:
host: postgres.postgres.svc.cluster.local
database: bindplane
username: postgres
password: password
maxConnections: 20

eventbus:
type: 'pubsub'
pubsub:
projectid: 'my-project'
topic: 'bindplane'
insecure: true
endpoint: "pubsub-emulator.pubsub.svc.cluster.local:8681"

replicas: 3

resources:
requests:
memory: 100Mi
cpu: 100m
limits:
memory: 100Mi
cpu: 100m
75 changes: 75 additions & 0 deletions test/helper/postgres/postgres.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
kind: Namespace
apiVersion: v1
metadata:
name: postgres
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: postgres
namespace: postgres
spec:
serviceName: postgres
selector:
matchLabels:
app: postgres
template:
metadata:
labels:
app: postgres
spec:
initContainers:
- name: take-data-dir-ownership
image: alpine:3
command:
- chown
- -R
- 70:70
- /data
volumeMounts:
- name: database
mountPath: /data
containers:
- image: postgres:14.7-alpine3.17
name: postgres
ports:
- containerPort: 5432
name: postgres
resources:
requests:
cpu: 100m
memory: 100Mi
limits:
memory: 100Mi
env:
- name: POSTGRES_PASSWORD
value: password
- name: PGDATA
value: /data/postgresql
volumeMounts:
- mountPath: /data
name: database
livenessProbe:
tcpSocket:
port: 5432
volumes:
- name: database
emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
labels:
app: postgres
name: postgres
namespace: postgres
spec:
ports:
- port: 5432
protocol: TCP
targetPort: postgres
name: postgres
selector:
app: postgres
sessionAffinity: None
type: ClusterIP
59 changes: 59 additions & 0 deletions test/helper/pubsub/pubsub.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
kind: Namespace
apiVersion: v1
metadata:
name: pubsub
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: pubsub-emulator
namespace: pubsub
spec:
serviceName: pubsub-emulator
selector:
matchLabels:
app: pubsub-emulator
template:
metadata:
labels:
app: pubsub-emulator
spec:
containers:
- image: messagebird/gcloud-pubsub-emulator:latest
name: emulator
ports:
- containerPort: 8681
- containerPort: 8682
resources:
requests:
cpu: 100m
memory: 512Mi
limits:
memory: 512Mi
env:
- name: PUBSUB_PROJECT1
value: bindplane-local-dev,test-topic:test-subscription
livenessProbe:
tcpSocket:
port: 8681
readinessProbe:
tcpSocket:
port: 8682
---
apiVersion: v1
kind: Service
metadata:
labels:
app: pubsub-emulator
name: pubsub-emulator
namespace: pubsub
spec:
ports:
- port: 8681
protocol: TCP
targetPort: 8681
name: emulator
selector:
app: pubsub-emulator
sessionAffinity: None
type: ClusterIP

0 comments on commit d380c0a

Please sign in to comment.