Test environment

[johannadm]

As agreed, DigitalOcean will be serving as the test environment. How should the Github workflows be set when using DigitalOcean?

[ddeboer]

We run a managed Kubernetes cluster on DigitalOcean (so DigitalOcean is an implementation detail, really).

Workflow

I suggest the following workflow:

• For each of the four components, you build an OCI container image when a commit is pushed to the main/master branch (example).
• You push that container to GHCR.
• The NDE infrastructure either pulls image updates (using Flux) or the GitHub Action pushes the update (by re-deploying).
• Your software follows best practices that make it work well in a container-based infrastructure, such as logs to stdout and configurable through env vars.

Questions

1. Should we move this repository to the NDE namespace first? This has two advantages:
   • as an admin on the repo, I would be able to configure DigitalOcean secrets without the risk that they leak;
   • the GHCR container name, which always includes the namespace, would remain stable, which is good for any third parties using the containers.
2. Will continuously deploying the main branch work for you? Or do you follow some GitFlow process where a separate branch or only tags must be deployed?
3. What external dependencies do the four components have? For example data stores or volumes and configuration parameters. The Solid Community Server is an obvious one, but there may be more. Even better, do you already run similar software on a container platform such as Kubernetes? If so, can we exchange some ideas?

[wouteraj]

Hi David,

Thanks for the feedback, sounds doable at first glance. I'll create an issue to make the necessary changes.

With regards to your questions:

Should we move this repository to the NDE namespace first?

We would prefer to develop in our own Github organization, because it has been integrated with our internal project management system (Wrike). That being said, we could do any of the following:

1. Keep developing on Digita's Github organization and push changes to a second repository in NDE's Github organization at the end of every sprint.
2. Keep developing on Digita's Github organization and transfer ownership of the repository to NDE's Github organization when the project is delivered.
3. Transfer the repository to NDE's Github organization and give us permissions to integrate our project management system within your organization.

That being said, we're open for other ideas as well :-)

Will continuously deploying the main branch work for you? Or do you follow some GitFlow process where a separate branch or only tags must be deployed?

For internal projects we usually work as follows.

• deploy to our shared development environment whenever a version tag is created
• deploy our production environment whenever the master branch is updated.

So, no problem!

What external dependencies do the four components have? For example data stores or volumes and configuration parameters. The Solid Community Server is an obvious one, but there may be more. Even better, do you already run similar software on a container platform such as Kubernetes? If so, can we exchange some ideas?

Very valid point. Even though the demo apps we are building are stateless, some other components of the demo environment are not. However, I would personally consider to configure these components to use an in-memory store, so we don't have to worry about volumes. My reasoning is as follows:

• The demo apps we are building, will be able to work with any (compliant) Solid server. This means you'll be able to use (among others) Inrupt PodSpaces which is persistent, as well as the Community Solid Server deployed in the demo environment.
• The main objective of deploying the Community Solid Server in the demo environment, would be to have a controlled environment to demo the apps. Using an in-memory store will ensure we will be able to easily re-deploy test data, which can be kept in version control.

[ddeboer]

That being said, we could do any of the following:

Let’s go with option 1, which disrupts both Digita’s and NDE’s existing processes as little as possible. This will make it harder to migrate the repository to NDE at the end of the project if we want to keep the issue and PR history.

I’ll fork the repository to the NDE organisation and you’ll push any changes to master to our fork too (optionally automated with a GitHub Action). Forking is currently disabled on the repository. Can you enable it?

An open question is how we can have the GitHub workflow that deploys to NDE infrastructure run only on the NDE repository’s main branch, not on the Digita one. I suggest we just use a job.if that checks an env var.

However, I would personally consider to configure these components to use an in-memory store, so we don't have to worry about volumes.

Agreed: let’s keep things simple and start with a demo environment that doesn’t require persistence (so needs no volumes). As you mentioned today, (external) test users will still have persistence because they’ll be using their own pod, hosted elsewhere.

The main objective of deploying the Community Solid Server in the demo environment, would be to have a controlled environment to demo the apps.

Doesn’t the fact that we combine in-memory with externally hosted user Solid pods mean that we do not need a Community Solid Server in our demo environment? Or would running that server still provide benefits?