Monorepo

Repo https://github.com/tgandrews/ons-monorepo

Current Problems

1. Making a change

Currently when working on author it often requires changes across a number of components to complete a change. This can create a dependency tree of pull requests which creates a number of problems:

• constant context switching
• need reviewers to look at lots of PRs
• reviewers need to pull all the changes to get it running locally
• PR CI is often red as the CI is using already released version of dependencies

2. Sharing is tricky

We have a few components that need to be shared (linting rules and schema) and currently we have a relatively complex process to make changes to these including releasing npm packages.

Proposed Solution

Overview

Import all the existing dependencies into one git repository like here: https://github.com/tgandrews/ons-monorepo

What is a monorepo?

1. https://danluu.com/monorepo/
2. https://hackernoon.com/one-vs-many-why-we-moved-from-multiple-git-repos-to-a-monorepo-and-how-we-set-it-up-f4abb0cfe469

Examples

1. https://github.com/withspectrum/spectrum This is quite interesting as they have hoisted all their dependencies to one top level package.json and then their components are all children inheriting them. This would make their docker images larger as they would contain unnecessary dependencies but would make managing the dependencies easier.
2. https://github.com/babel/babel This is not such a useful example as it is a project focussed on released lots of npm packages which does not align with our use case well.

How does this help?

1. A change can be made across all the necessary components in one go without repository overhead.
2. Shared resources can easily be extracted with no unnecessary overhead of having to release npm packages.
3. PR should always be green as they contain everything they are changing.
4. A reviewer can checkout one PR and run everything locally.

Costs

1. Loss of git history - looks like some tools might help with this.
2. Bigger PRs - but fewer of them 3 PRs x 100 changes vs 1 PR x 300 changes.

Spike investigation

CI

Currently we run all the tests for a given repo in a PR but this is not ideal if we have all components in the project. So implemented a travis matrix in combination with a shell script to determine if a component is in the changeset for the change. This works for the simple set up but will need to get more complex if a shared folder is changed as this could impact all components. Additionally we could run the end-to-end tests for every change as well the component tests to give us greater level of confidence.

• .travis.yml
• build-conditions.sh

Netlify

For author UI changes we spin up netlify environments for both storybook and the app. This makes it easier for reviewers (including designers) to review the changes so we need it to work in a monorepo environment. Netlify allows you to provide a .toml configuration file where we can specify the location of the project and the command to run.

• netlify.toml

CD

Currently our CD pipeline is triggered on a commit to master of a repository. We would have to change this to all look at the same repository then run docker build in the necessary component folder.

Tooling

• Lerna - This is a tool used by babel to manage their huge number of packages and the dependencies between them. However, it is not appropriate for us as our release artefacts are docker images and not npm packages. We are only using them because of our current repo structure at the moment.
• Yarn workspaces - Again this looks like a great tool for managing projects releasing lots of npm packages so is not appropriate as it currently creates just one top level yarn.lock which does not align well with our current docker build.