[bitnami/mlflow] Add support for Google Cloud Storage as artifact sto…

…re (bitnami#28938) * Add support for GCS artifact storage Signed-off-by: Nicolas Mauti <[email protected]> * Generate README from values file Signed-off-by: Nicolas Mauti <[email protected]> * Update CHANGELOG.md Signed-off-by: Bitnami Containers <[email protected]> * Update CHANGELOG.md Signed-off-by: Bitnami Containers <[email protected]> * Update CHANGELOG.md Signed-off-by: Bitnami Containers <[email protected]> * Apply suggestions from code review Co-authored-by: Andrés Bono <[email protected]> Signed-off-by: Nicolas Mauti <[email protected]> * Fix keys.json mounting Signed-off-by: Nicolas Mauti <[email protected]> * Update CHANGELOG.md Signed-off-by: Bitnami Containers <[email protected]> * Apply suggestions from code review Co-authored-by: Andrés Bono <[email protected]> Signed-off-by: Nicolas Mauti <[email protected]> * Update CHANGELOG.md Signed-off-by: Bitnami Containers <[email protected]> * Update README.md with readme-generator-for-helm Signed-off-by: Bitnami Containers <[email protected]> --------- Signed-off-by: Nicolas Mauti <[email protected]> Signed-off-by: Bitnami Containers <[email protected]> Signed-off-by: Carlos Rodríguez Hernández <[email protected]> Co-authored-by: Bitnami Containers <[email protected]> Co-authored-by: Carlos Rodríguez Hernández <[email protected]> Co-authored-by: Andrés Bono <[email protected]> Signed-off-by: Michael Faherty <[email protected]>
mike12806 · Oct 30, 2024 · 8c31ca5 · 8c31ca5
1 parent cc20726
commit 8c31ca5
Show file tree

Hide file tree

Showing 6 changed files with 79 additions and 3 deletions.
diff --git a/bitnami/mlflow/CHANGELOG.md b/bitnami/mlflow/CHANGELOG.md
@@ -1,8 +1,12 @@
 # Changelog
 
-## 1.4.25 (2024-08-31)
+## 1.5.0 (2024-09-05)
 
-* [bitnami/mlflow] Release 1.4.25 ([#29139](https://github.com/bitnami/charts/pull/29139))
+* [bitnami/mlflow] Add support for Google Cloud Storage as artifact store ([#28938](https://github.com/bitnami/charts/pull/28938))
+
+## <small>1.4.25 (2024-08-31)</small>
+
+* [bitnami/mlflow] Release 1.4.25 (#29139) ([07365a4](https://github.com/bitnami/charts/commit/07365a40a3e0246fdc926b421893782b9f17ccd7)), closes [#29139](https://github.com/bitnami/charts/issues/29139)
 
 ## <small>1.4.24 (2024-08-26)</small>
 

diff --git a/bitnami/mlflow/Chart.yaml b/bitnami/mlflow/Chart.yaml
@@ -44,4 +44,4 @@ sources:
 - https://github.com/bitnami/charts/tree/main/bitnami/mlflow
 - https://github.com/bitnami/containers/tree/main/bitnami/mlflow
 - https://github.com/mlflow/mlflow
-version: 1.4.25
+version: 1.5.0
diff --git a/bitnami/mlflow/README.md b/bitnami/mlflow/README.md
@@ -460,6 +460,17 @@ The command deploys mlflow on the Kubernetes cluster in the default configuratio
 | `externalS3.bucket`                       | External S3 bucket                                                                                                                                                          | `mlflow`        |
 | `externalS3.serveArtifacts`               | Whether artifact serving is enabled                                                                                                                                         | `true`          |
 
+### External Google Cloud Storage parameters
+
+| Name                                 | Description                                                                                                               | Value   |
+| ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------- | ------- |
+| `externalGCS.bucket`                 | GCS bucket name. Activate gcs artifact storage if set                                                                     | `""`    |
+| `externalGCS.googleCloudProject`     | Google Cloud Project to use (optional, needed when using "default application credentials")                               | `""`    |
+| `externalGCS.useCredentialsInSecret` | Whether to read the GCS application credentials from a secret                                                             | `false` |
+| `externalGCS.existingSecret`         | Name of an existing secret key containing the application credentials file (required when useCredentialsInSecret is true) | `""`    |
+| `externalGCS.existingSecretKey`      | Key in the existing secret containing the application credentials (required when useCredentialsInSecret is true)          | `""`    |
+| `externalGCS.serveArtifacts`         | Whether artifact serving is enabled                                                                                       | `true`  |
+
 The MLflow chart supports three different ways to load your files in the `run` deployment. In order of priority, they are:
 
 1. Existing config map

diff --git a/bitnami/mlflow/templates/_helpers.tpl b/bitnami/mlflow/templates/_helpers.tpl
@@ -544,13 +544,19 @@ Return the volume-permissions init container
       mountPath: /tmp
 {{- end -}}
 
+
+{{/*
+Deal with external artifact storage
+*/}}
+
 {{/*
 Return MinIO(TM) fullname
 */}}
 {{- define "mlflow.v0.minio.fullname" -}}
 {{- include "common.names.dependency.fullname" (dict "chartName" "minio" "chartValues" .Values.minio "context" $) -}}
 {{- end -}}
 
+
 {{/*
 Return whether S3 is enabled
 */}}
@@ -648,6 +654,15 @@ Return the S3 secret access key inside the secret
     {{- end -}}
 {{- end -}}
 
+{{/*
+Return whether GCS is enabled
+*/}}
+{{- define "mlflow.v0.gcs.enabled" -}}
+    {{- if and (not .Values.minio.enabled) (not .Values.externalS3.host) .Values.externalGCS.bucket -}}
+        {{- true }}
+    {{- end -}}
+{{- end -}}
+
 
 {{/*
 Return the proper git image name

diff --git a/bitnami/mlflow/templates/tracking/deployment.yaml b/bitnami/mlflow/templates/tracking/deployment.yaml
@@ -141,12 +141,17 @@ spec:
             {{- end }}
             {{- if (include "mlflow.v0.s3.enabled" .) }}
             - --artifacts-destination=s3://{{ include "mlflow.v0.s3.bucket" . }}
+            {{- else if (include "mlflow.v0.gcs.enabled" .) }}
+            - --artifacts-destination=gs://{{ .Values.externalGCS.bucket }}
             {{- else }}
             - --artifacts-destination={{ .Values.tracking.persistence.mountPath }}/mlartifacts
             {{- end }}
             {{- if and (not (include "mlflow.v0.s3.serveArtifacts" .)) (include "mlflow.v0.s3.enabled" .) }}
             - --default-artifact-root=s3://{{ include "mlflow.v0.s3.bucket" . }}
             - --no-serve-artifacts
+            {{- else if and (not .Values.externalGCS.serveArtifacts) (include "mlflow.v0.gcs.enabled" .) }}
+            - --default-artifact-root=gs://{{ .Values.externalGCS.bucket }}
+            - --no-serve-artifacts
             {{ else }}
             - --serve-artifacts
             {{- end }}
@@ -190,6 +195,16 @@ spec:
             - name: MLFLOW_S3_ENDPOINT_URL
               value: {{ printf "%s://%s:%v" (include "mlflow.v0.s3.protocol" .) (include "mlflow.v0.s3.host" .) (include "mlflow.v0.s3.port" .) | quote }}
             {{- end }}
+            {{- if (include "mlflow.v0.gcs.enabled" .) }}
+            {{- if .Values.externalGCS.useCredentialsInSecret }}
+            - name: GOOGLE_APPLICATION_CREDENTIALS
+              value: /bitnami/gcs/key.json
+            {{- end }}
+            {{- if .Values.externalGCS.googleCloudProject }}
+            - name: GOOGLE_CLOUD_PROJECT
+              value: {{ .Values.externalGCS.googleCloudProject }}
+            {{- end }}
+            {{- end }}
             {{- if .Values.tracking.extraEnvVars }}
             {{- include "common.tplvalues.render" (dict "value" .Values.tracking.extraEnvVars "context" $) | nindent 12 }}
             {{- end }}
@@ -273,6 +288,11 @@ spec:
               {{- if .Values.tracking.persistence.subPath }}
               subPath: {{ .Values.tracking.persistence.subPath }}
               {{- end }}
+            {{- if and (include "mlflow.v0.gcs.enabled" .) .Values.externalGCS.useCredentialsInSecret }}
+            - name: gcs-key
+              mountPath: /bitnami/gcs/key.json
+              subPath: key.json
+            {{- end }}
           {{- if .Values.tracking.extraVolumeMounts }}
           {{- include "common.tplvalues.render" (dict "value" .Values.tracking.extraVolumeMounts "context" $) | nindent 12 }}
           {{- end }}
@@ -310,4 +330,12 @@ spec:
         {{- if .Values.tracking.extraVolumes }}
         {{- include "common.tplvalues.render" (dict "value" .Values.tracking.extraVolumes "context" $) | nindent 8 }}
         {{- end }}
+        {{- if and (include "mlflow.v0.gcs.enabled" .) .Values.externalGCS.useCredentialsInSecret }}
+        - name: gcs-key
+          secret:
+            secretName: {{ .Values.externalGCS.existingSecret }}
+            items:
+              - key: {{ .Values.externalGCS.existingSecretKey }}
+                path: key.json
+        {{- end }}
 {{- end }}
diff --git a/bitnami/mlflow/values.yaml b/bitnami/mlflow/values.yaml
@@ -1395,6 +1395,7 @@ minio:
     loadBalancerIP: ""
     ports:
       api: 80
+
 ## @section External S3 parameters
 ## All of these values are only used when minio.enabled is set to false
 ## @param externalS3.host External S3 host. When using AWS S3, include appropriate [regional code](https://docs.aws.amazon.com/general/latest/gr/s3.html#s3_region), e.g. "eu-central-1.amazonaws.com
@@ -1421,3 +1422,20 @@ externalS3:
   protocol: "https"
   bucket: "mlflow"
   serveArtifacts: true
+
+## @section External Google Cloud Storage parameters
+## All of these values are only used when minio.enabled is set to false and externalS3 is not configured (host is empty)
+## @param externalGCS.bucket GCS bucket name. Activate gcs artifact storage if set
+## @param externalGCS.googleCloudProject Google Cloud Project to use (optional, needed when using "default application credentials")
+## @param externalGCS.useCredentialsInSecret Whether to read the GCS application credentials from a secret
+## @param externalGCS.existingSecret Name of an existing secret key containing the application credentials file (required when useCredentialsInSecret is true)
+## @param externalGCS.existingSecretKey Key in the existing secret containing the application credentials (required when useCredentialsInSecret is true)
+## @param externalGCS.serveArtifacts Whether artifact serving is enabled
+##
+externalGCS:
+  bucket: ""
+  googleCloudProject: ""
+  useCredentialsInSecret: false
+  existingSecret: ""
+  existingSecretKey: ""
+  serveArtifacts: true