tag 0.1.0

EOEPCA · Oct 7, 2024 · 9af4cff · 9af4cff
1 parent 5b88bc3
commit 9af4cff
Show file tree

Hide file tree

Showing 6 changed files with 210 additions and 60 deletions.
diff --git a/docs/design/motivation.md b/docs/design/motivation.md
@@ -12,3 +12,50 @@ The v2 version of the workspace mitigates these issues:
 * The workspace controller automatically creates a dedicated space in a Git repository (either a new repository or a separate folder) for each workspace, allowing for declarative descriptions of all deployed services with their current software version and configuration, so this information can be easily inspected. Custom services can be installed via GitOps mechanisms and are declaratively stated the same way. 
 * The provisioning pipeline is declaratively stated and can be adapted by the operator. Best-practise blueprints are shared in the workspace EOEPCA repository. The provisioning pipeline is continuously reconciled, always trying to match the desired state with the observed state. Updates of services can therefore be selectively or globally triggered via Git.
 * The vcluster tooling is automatically deployed per namespace, providing a dedicated Kubernetes API server runtime for each workspace.
+
+
+PackgeR - bundle files/objects together for distribution, deployment, or storage.
+
+Define Workspace CRD specification allowing the Workspace controller to:
+- Establish a dedicated namespace for each project/user in the Host Kubernetes cluster.
+- Apply Kubernetes policies such as ResourceQuota, LimitRange, and NetworkPolicy to the namespace.
+- Deploy a vCluster with best-practice configurations within the namespace.
+- Create a new Git repository (or a folder in an existing Git repository, depending on the global setup) to store the desired manifests for - Workspace Services to be reconciled through Flux GitOps principles.
+- Connect Flux to reconcile the Git repository (or folder) with the vCluster.
+- Implement Kubernetes Validating Webhooks to enforce bucket creation policies, such as maximum number and size, and naming pattern conventions, within the namespace.
+
+Reconcile [Workspace CRDs](https://github.com/EOEPCA/workspace/issues/1) and expose:
+- vCluster credentials for direct “virtual” Kubernetes cluster access,
+- Git settings used for Flux,
+- The current reconciliation state.
+
+
+Define Storage CRD specification allowing the Storage controller to:
+- Establish a dedicated bucket or connect a federated bucket.
+- Configure bucket access and CORS policies for the bucket.
+
+Reconcile [Storage CRDs](https://github.com/EOEPCA/workspace/issues/3) and expose:
+- Bucket credentials,
+- Access URL,
+- The current reconciliation state.
+
+- 
+- establish a dedicated namespace for each project/user in the Host Kubernetes cluster.
+apply ResourceQuota to namespace
+create Storage via a Storage CRD (https://github.com/EOEPCA/workspace/issues/3) to be picked up by Storage Controller (https://github.com/EOEPCA/workspace/issues/4)
+deploy Workspace UI (https://github.com/EOEPCA/workspace/issues/7)
+
+
+
+- endpoints to 
+
+
+Physical storage may either be dynamically provisioned (see [Storage CRDs](https://github.com/EOEPCA/workspace/issues/3)) or linked to existing.
+
+Allow users to search and browse through the content of the storage buckets connected to a given workspace using directory-based file system semantics. 
+
+Allow sharing of specific content via pre-signed URLs, enabling direct HTTP access without requiring user authentication.
+
+--
+
+utili
diff --git a/meetings.md b/meetings.md
@@ -1,9 +1,23 @@
+# Agenda 20241001
+
+- trimmed down workspace-api for v2.0, see Compatibility matrix (https://github.com/EOEPCA/workspace/issues/24)
+
+- clean deployments of workspace components to follow, there will be one global Workspace UI for demonstration and each user/team will have its own dedicated Workspace UI
+
+- present slides and sketch demo for Q2/2024 review, Q3/2024 outlook
+
 # Agenda 20240917
 
 - aligned on process how CRDs can be rolled out on cluster by making them part of ArgoCD deployment
 
 - pipeline adapted to rollout latest version of Workspace UI with all connected buckets to a workspace
 
+> [!Note]
+> Upcoming:
+> - link external buckets via Storage Layer API (and subsequently expose capabilities in Workspace UI)
+> - Storage Layer API endpoint documentation
+
+
 # Agenda 20240903
 
 ## Workspace Concepts and API Interfaces

diff --git a/setup/eoepca-demo/storage/composition.yaml b/setup/eoepca-demo/storage/composition.yaml
@@ -3,6 +3,7 @@ kind: Composition
 metadata:
   name: storage
 spec:
+  writeConnectionSecretsToNamespace: workspace
   compositeTypeRef:
     apiVersion: epca.eo/v1beta1
     kind: XStorage
@@ -15,7 +16,7 @@ spec:
       apiVersion: pt.fn.crossplane.io/v1beta1
       kind: Resources
       resources:
-      - name: bucket-results
+      - name: bucket
         base:
           apiVersion: minio.crossplane.io/v1
           kind: Bucket
@@ -29,11 +30,6 @@ spec:
         - type: FromCompositeFieldPath
           fromFieldPath: "spec.claimRef.namespace"
           toFieldPath: "metadata.name"
-          transforms:
-          - type: string
-            string:
-              type: Format
-              fmt: '%s-results'
       - name: bucket-stage
         base:
           apiVersion: minio.crossplane.io/v1
@@ -53,7 +49,7 @@ spec:
             string:
               type: Format
               fmt: '%s-stage'
-      - name: policy-results
+      - name: policy
         base:
           apiVersion: minio.crossplane.io/v1
           kind: Policy
@@ -66,19 +62,9 @@ spec:
         - type: FromCompositeFieldPath
           fromFieldPath: "spec.claimRef.namespace"
           toFieldPath: "metadata.name"
-          transforms:
-          - type: string
-            string:
-              type: Format
-              fmt: '%s-results'
         - type: FromCompositeFieldPath
           fromFieldPath: "spec.claimRef.namespace"
           toFieldPath: "spec.forProvider.allowBucket"
-          transforms:
-          - type: string
-            string:
-              type: Format
-              fmt: '%s-results'
       - name: policy-stage
         base:
           apiVersion: minio.crossplane.io/v1
@@ -116,22 +102,17 @@ spec:
             providerConfigRef:
               name: provider-minio
             writeConnectionSecretToRef:
-              name: bucket
+              namespace: workspace
         patches:
         - type: FromCompositeFieldPath
           fromFieldPath: "spec.claimRef.namespace"
           toFieldPath: "metadata.name"
         - type: FromCompositeFieldPath
           fromFieldPath: "spec.claimRef.namespace"
-          toFieldPath: "spec.writeConnectionSecretToRef.namespace"
+          toFieldPath: "spec.writeConnectionSecretToRef.name"
         - type: FromCompositeFieldPath
           fromFieldPath: "spec.claimRef.namespace"
           toFieldPath: "spec.forProvider.policies[0]"
-          transforms:
-          - type: string
-            string:
-              type: Format
-              fmt: '%s-results'
         - type: FromCompositeFieldPath
           fromFieldPath: "spec.claimRef.namespace"
           toFieldPath: "spec.forProvider.policies[1]"
@@ -155,4 +136,4 @@ spec:
           fromConnectionSecretKey: AWS_SECRET_ACCESS_KEY          
         - name: bucketname
           type: FromFieldPath
-          fromFieldPath: spec.claimRef.namespace
+          fromFieldPath: metadata.name
diff --git a/setup/eoepca-demo/workspace/composition.yaml b/setup/eoepca-demo/workspace/composition.yaml
@@ -32,7 +32,7 @@ spec:
               name: provider-kubernetes
         patches:
         - type: FromCompositeFieldPath
-          fromFieldPath: "metadata.name"
+          fromFieldPath: "spec.claimRef.name"
           toFieldPath: "spec.forProvider.manifest.metadata.name"
       - name: resourcequota
         base:
@@ -51,11 +51,8 @@ spec:
             providerConfigRef:
               name: provider-kubernetes
         patches:
-        #- type: FromCompositeFieldPath
-        #  fromFieldPath: "metadata.name"
-        #  toFieldPath: "spec.forProvider.manifest.metadata.name"
         - type: FromCompositeFieldPath
-          fromFieldPath: "metadata.name"
+          fromFieldPath: "spec.claimRef.name"
           toFieldPath: "spec.forProvider.manifest.metadata.namespace"
         - type: FromCompositeFieldPath
           fromFieldPath: "spec.subscription"
@@ -96,14 +93,13 @@ spec:
                 spec:
                   location: EU
                   acl: private
+                  writeConnectionSecretToRef:
+                    name: bucket
             providerConfigRef:
               name: provider-kubernetes
         patches:
-        #- type: FromCompositeFieldPath
-        #  fromFieldPath: "metadata.name"
-        #  toFieldPath: "spec.forProvider.manifest.metadata.name"
         - type: FromCompositeFieldPath
-          fromFieldPath: "metadata.name"
+          fromFieldPath: "spec.claimRef.name"
           toFieldPath: "spec.forProvider.manifest.metadata.namespace"
       - name: deployment-workspace-ui
         base:
@@ -139,7 +135,7 @@ spec:
                     spec:
                       containers:
                       - name: workspace-ui
-                        image: ghcr.io/versioneer-tech/package-r:v1.2.1
+                        image: ghcr.io/versioneer-tech/package-r:v1.2.5
                         ports:
                         - name: http
                           containerPort: 80
@@ -152,17 +148,19 @@ spec:
                         - name: AWS_ACCESS_KEY_ID
                           valueFrom:
                             secretKeyRef:
-                              name: bucket-credentials
-                              key: AWS_ACCESS_KEY_ID
+                              name: bucket
+                              key: access
                         - name: AWS_SECRET_ACCESS_KEY
                           valueFrom:
                             secretKeyRef:
-                              name: bucket-credentials
-                              key: AWS_SECRET_ACCESS_KEY
+                              name: bucket
+                              key: secret
                         - name: AWS_REGION
                           value: eu-central-1
                         - name: AWS_ENDPOINT_URL
-                          value: https://minio.develop.eoepca.org                        
+                          value: https://minio.develop.eoepca.org
+                        - name: PASSWORD
+                          value: "changeme"
                         resources:
                           limits:
                             cpu: 1
@@ -174,14 +172,11 @@ spec:
             providerConfigRef:
               name: provider-kubernetes
         patches:
-        #- type: FromCompositeFieldPath
-        #  fromFieldPath: "metadata.name"
-        #  toFieldPath: "spec.forProvider.manifest.metadata.name"
         - type: FromCompositeFieldPath
-          fromFieldPath: "metadata.name"
+          fromFieldPath: "spec.claimRef.name"
           toFieldPath: "spec.forProvider.manifest.metadata.namespace"
         - type: FromCompositeFieldPath
-          fromFieldPath: "metadata.name"
+          fromFieldPath: "spec.claimRef.name"
           toFieldPath: "spec.forProvider.manifest.spec.template.spec.containers[0].env[0].value"
           transforms:
           - type: string
@@ -213,13 +208,131 @@ spec:
             providerConfigRef:
               name: provider-kubernetes
         patches:
-        #- type: FromCompositeFieldPath
-        #  fromFieldPath: "metadata.name"
-        #  toFieldPath: "spec.forProvider.manifest.metadata.name"
         - type: FromCompositeFieldPath
-          fromFieldPath: "metadata.name"
+          fromFieldPath: "spec.claimRef.name"
           toFieldPath: "spec.forProvider.manifest.metadata.namespace"
-      # - name: github TODO
-      # - name: vcluster TODO
-      # - name: gitops TODO
-      # - name: validation webhook TODO
+      - name: ingress-workspace-ui
+        base:
+          apiVersion: kubernetes.crossplane.io/v1alpha2
+          kind: Object
+          metadata: {}
+          spec:
+            forProvider:
+              manifest:
+                apiVersion: networking.k8s.io/v1
+                kind: Ingress
+                metadata:
+                  name: workspace-ui
+                  annotations:
+                    cert-manager.io/cluster-issuer: letsencrypt-prod
+                    kubernetes.io/ingress.class: nginx
+                    nginx.ingress.kubernetes.io/enable-cors: "true"
+                    nginx.ingress.kubernetes.io/proxy-read-timeout: "600"
+                spec:
+                  rules:
+                  - host: <replaced>
+                    http:
+                      paths:
+                      - backend:
+                          service:
+                            name: workspace-ui
+                            port:
+                              name: http
+                        path: /
+                        pathType: ImplementationSpecific
+                  tls:
+                  - hosts:
+                    - <replaced>
+                    secretName: workspace-ui-tls               
+            providerConfigRef:
+              name: provider-kubernetes
+        patches:
+        - type: FromCompositeFieldPath
+          fromFieldPath: "spec.claimRef.name"
+          toFieldPath: "spec.forProvider.manifest.metadata.namespace"
+        - type: FromCompositeFieldPath
+          fromFieldPath: "spec.claimRef.name"
+          toFieldPath: "spec.forProvider.manifest.spec.rules[0].host"
+          transforms:
+          - type: string
+            string:
+              type: Format
+              fmt: '%s.develop.eoepca.org'
+        - type: FromCompositeFieldPath
+          fromFieldPath: "spec.claimRef.name"
+          toFieldPath: "spec.forProvider.manifest.spec.tls[0].hosts[0]"
+          transforms:
+          - type: string
+            string:
+              type: Format
+              fmt: '%s.develop.eoepca.org'
+      - name: source-workspace-ui
+        base:
+          apiVersion: kubernetes.crossplane.io/v1alpha2
+          kind: Object
+          metadata: {}
+          spec:
+            forProvider:
+              manifest:
+                apiVersion: package.r/alphav1
+                kind: Source
+                spec:
+                  access:               
+                    secretName: bucket
+            providerConfigRef:
+              name: provider-kubernetes
+        patches:
+        - type: FromCompositeFieldPath
+          fromFieldPath: "spec.claimRef.name"
+          toFieldPath: "spec.forProvider.manifest.metadata.name"
+        - type: FromCompositeFieldPath
+          fromFieldPath: "spec.claimRef.name"
+          toFieldPath: "spec.forProvider.manifest.metadata.namespace"
+      - name: source-global-workspace-ui
+        base:
+          apiVersion: kubernetes.crossplane.io/v1alpha2
+          kind: Object
+          metadata: {}
+          spec:
+            forProvider:
+              manifest:
+                apiVersion: package.r/alphav1
+                kind: Source
+                metadata:
+                  namespace: workspace
+            providerConfigRef:
+              name: provider-kubernetes
+        patches:
+        - type: FromCompositeFieldPath
+          fromFieldPath: "spec.claimRef.name"
+          toFieldPath: "spec.forProvider.manifest.metadata.name"
+      - name: rolebinding-workspace-ui
+        base:
+          apiVersion: kubernetes.crossplane.io/v1alpha2
+          kind: Object
+          metadata: {}
+          spec:
+            forProvider:
+              manifest:
+                apiVersion: rbac.authorization.k8s.io/v1
+                kind: RoleBinding                
+                metadata:
+                  name: default-view
+                subjects:
+                - kind: ServiceAccount
+                  name: default
+                  namespace: <replaced>
+                roleRef:
+                  kind: ClusterRole
+                  name: view
+                  apiGroup: rbac.authorization.k8s.io
+            providerConfigRef:
+              name: provider-kubernetes
+        patches:
+        - type: FromCompositeFieldPath
+          fromFieldPath: "spec.claimRef.name"
+          toFieldPath: "spec.forProvider.manifest.metadata.namespace"
+        - type: FromCompositeFieldPath
+          fromFieldPath: "spec.claimRef.name"
+          toFieldPath: "spec.forProvider.manifest.subjects[0].namespace"
+      # extend for github/gitlab, vcluster, ...