diff --git a/docs/nx-cloud/tutorial/circle-create-pr.avif b/docs/nx-cloud/tutorial/circle-create-pr.avif new file mode 100644 index 0000000000000..e90378d9c6634 Binary files /dev/null and b/docs/nx-cloud/tutorial/circle-create-pr.avif differ diff --git a/docs/nx-cloud/tutorial/circle-new-run.avif b/docs/nx-cloud/tutorial/circle-new-run.avif new file mode 100644 index 0000000000000..a43599929b5df Binary files /dev/null and b/docs/nx-cloud/tutorial/circle-new-run.avif differ diff --git a/docs/nx-cloud/tutorial/circle-pr.avif b/docs/nx-cloud/tutorial/circle-pr.avif new file mode 100644 index 0000000000000..6bfb161541a22 Binary files /dev/null and b/docs/nx-cloud/tutorial/circle-pr.avif differ diff --git a/docs/nx-cloud/tutorial/circle-setup-project.avif b/docs/nx-cloud/tutorial/circle-setup-project.avif new file mode 100644 index 0000000000000..e785381610659 Binary files /dev/null and b/docs/nx-cloud/tutorial/circle-setup-project.avif differ diff --git a/docs/nx-cloud/tutorial/circle.md b/docs/nx-cloud/tutorial/circle.md index e93e8288c0ecc..d98492c6da0bc 100644 --- a/docs/nx-cloud/tutorial/circle.md +++ b/docs/nx-cloud/tutorial/circle.md @@ -5,14 +5,12 @@ description: In this tutorial you'll set up continuous integration with Circle C # Circle CI with Nx -In this tutorial we're going to learn how to leverage Nx to setup a scalable CI pipeline on Circle CI. You're going to learn +In this tutorial we're going to learn how to leverage Nx to setup a scalable CI pipeline on Circle CI. As repositories get bigger, making sure that the CI is fast, reliable and maintainable can get very challenging. Nx provides a solution. -- how to set up Circle CI and configure Nx -- how to run tasks for only the projects that were affected by a given PR -- how to enable remote caching -- how to parallelize and distribute tasks across multiple machines - -Note, many of these optimizations are incremental, meaning you could set up running tasks for only affected projects and stop there. Later when you experience slow CI runs, you could add caching to further improve CI performance or even go further and distribute tasks across machines. +- Nx reduces wasted time in CI with the [`affected` command](/ci/features/affected). +- Nx Replay's [remote caching](/ci/features/remote-cache) will reuse task artifacts from different CI executions making sure you will never run the same computation twice. +- Nx Agents [efficiently distribute tasks across machines](/ci/concepts/parallelization-distribution) ensuring constant CI time regardless of the repository size. The right number of machines is allocated for each PR to ensure good performance without wasting compute. +- Nx Atomizer [automatically splits](/ci/features/split-e2e-tasks) large e2e tests to distribute them across machines. Nx can also automatically [identify and rerun flaky e2e tests](/ci/features/flaky-tasks). ## Example Repository @@ -31,7 +29,7 @@ To get started: 1. [Fork the nx-shop repo](https://github.com/nrwl/nx-shops/fork) and then clone it to your local machine ```shell - git clone git@github.com:/nx-shops.git + git clone https://github.com//nx-shops.git ``` 2. Install dependencies (this repo uses [PNPM](https://pnpm.io/) but you should be able to also use any other package manager) @@ -40,152 +38,78 @@ To get started: pnpm i ``` -3. Explore the structure of the repo using **the Nx Graph** - - ```shell - pnpm nx graph - ``` - -4. Finally, make sure all task are working on your machine, by running lint, test, build and e2e on all projects of the workspace +3. Make sure all tasks are working on your machine, by running lint, test, build and e2e on all projects of the workspace ```shell - pnpm nx run-many -t lint test build e2e-ci + pnpm nx run-many -t lint test build ``` -## Set Up Circle CI - -In order to use Circle CI, you need to [sign up and create an organization](https://circleci.com/docs/first-steps/#sign-up-and-create-an-org). Follow the steps in the Circle CI documentation to connect to your GitHub repository. When you are asked to configure a pipeline, choose any option, since we'll overwrite it in the next step. +## Connect to Circle CI -To verify that Circle CI is set up correctly we'll create a pipeline that just logs a message. First, checkout a new branch: - -```shell -git checkout -b circle-message -``` +In order to use Circle CI, you need to [sign up and create an organization](https://circleci.com/docs/first-steps/#sign-up-and-create-an-org). Follow the steps in the Circle CI documentation to connect to your GitHub repository to a project. -Then create (or modify) the `.circleci/config.yml` file with these contents: +![](/nx-cloud/tutorial/circle-setup-project.avif) -```yaml {% fileName=".circleci/config.yml" %} -version: 2.1 +The easiest way is to create a branch and PR in your GitHub repository. Note that a sample pipeline workflow file will be created, which we will overwrite in the next step. -jobs: - main: - docker: - - image: cimg/node:lts-browsers - steps: - - run: - name: Print a message - command: echo "Hello Circle CI!" +![](/nx-cloud/tutorial/circle-create-pr.avif) -workflows: - version: 2 +Once the PR is created, merge it into your main branch. - ci: - jobs: - - main -``` +![](/nx-cloud/tutorial/circle-pr.avif) -Next, commit this change, push the branch and create a PR on your forked GitHub repository: +And pull the changes locally: ```shell -git commit -am "pipeline that logs a message" -git push -u origin HEAD +git pull ``` -If everything was set up correctly, you should see a message from Circle CI in the PR with a success status. - -![All checks have passed in the PR](/nx-cloud/tutorial/Circle%20PR%20passed.png) +## Create a CI Workflow -Click on the job details and you should see the `Hello Circle CI` message in the logs. - -![The "Hello Circle CI" message is printed in the logs](/nx-cloud/tutorial/Message%20Logged.png) - -Merge your PR into the `main` branch when you're ready to move to the next section. - -## Configure Nx on Circle CI - -Now let's use Nx in the pipeline. The simplest way to use Nx is to run a single task, so we'll start by building our `cart` application. +First, we'll create a new branch to start adding a CI workflow. ```shell -pnpm nx build cart +git checkout -b setup-ci ``` -We need to adjust a couple of things on our CI pipeline to make this work: +Now we can use an Nx generator to create a default CI workflow file. -- clone the repository -- install NPM dependencies (in our nx-shop using PNPM) -- use Nx to run the `build` command +```shell +pnpm nx generate ci-workflow --ci=circleci +``` -Nx is an [npm package](https://www.npmjs.com/package/nx) so once NPM packages are installed we will be able to use it. +This generator will overwrite Circle CI's default `.circleci/config.yml` file to create a CI pipeline that will run the `lint`, `test`, `build` and `e2e` tasks for projects that are affected by any given PR. -Create a new branch called `build-one-app` and paste this code into the Circle CI config. +The key lines in the CI pipeline are highlighted in this excerpt: -```yaml {% fileName=".circleci/config.yml" highlightLines=["8-14"] %} +```yml {% fileName=".circleci/config.yml" highlightLines=["25-27"] %} version: 2.1 +orbs: + nx: nrwl/nx@1.6.2 + jobs: main: docker: - image: cimg/node:lts-browsers steps: - checkout - - run: - name: install dependencies - command: pnpm install --frozen-lockfile - - run: - name: Run build - command: pnpm nx build cart - -workflows: - version: 2 - - ci: - jobs: - - main -``` - -Once `node_modules` are in place, you can run normal Nx commands. In this case, we run `pnpm nx build cart`. Push the changes to your repository by creating a new PR and verifying the new CI pipeline correctly builds our application. - -![Building a single app with nx](/nx-cloud/tutorial/circle-single-build-success.jpg) - -You might have noticed that there's also a build running for `shared-header`, `shared-product-types` and `shared-product-ui`. These are projects in our workspace that `cart` depends on. Thanks to the [Nx task pipeline](/concepts/task-pipeline-configuration), Nx knows that it needs to build these projects first before building `cart`. This already helps us simplify our pipeline as we - -- don't need to define these builds automatically -- don't need to make any changes to our pipeline as our `cart` app grows and depends on more projects -- don't need to worry about the order of the builds - -Merge your PR into the `main` branch when you're ready to move to the next section. - -## Optimize our CI by caching NPM dependencies -While this isn't related to Nx specifically, it's a good idea to cache NPM dependencies in CI. This will speed up the CI pipeline by avoiding downloading the same dependencies over and over again. Circle CI has [a docs page on how to cache NPM dependencies](https://circleci.com/docs/caching/). + - run: + name: Install PNPM + command: npm install --prefix=$HOME/.local -g pnpm@8 -Adjust your CI pipeline script as follows + # Connect your workspace on nx.app and uncomment this to enable task distribution. + # The "--stop-agents-after" is optional, but allows idle agents to shut down once the "e2e-ci" targets have been requested + # - run: pnpm dlx nx-cloud start-ci-run --distribute-on="5 linux-medium-js" --stop-agents-after="e2e-ci" -```yaml {% fileName=".circleci/config.yml" highlightLines=["10-11", "17-21"] %} -version: 2.1 + - run: pnpm install --frozen-lockfile + - nx/set-shas: + main-branch-name: 'main' -jobs: - main: - docker: - - image: cimg/node:lts-browsers - steps: - - checkout - # look for existing cache and restore if found - - restore_cache: - key: npm-dependencies-{{ checksum "pnpm-lock.yaml" }} - # install dependencies - - run: - name: install dependencies - command: pnpm install --frozen-lockfile - # save any changes to the cache - - save_cache: - key: npm-dependencies-{{ checksum "pnpm-lock.yaml" }} - paths: - - node_modules - - ~/.cache/Cypress # needed for the Cypress binary - - run: - name: Run build - command: pnpm nx build cart + # Prepend any command with "nx-cloud record --" to record its logs to Nx Cloud + # - run: pnpm exec nx-cloud record -- echo Hello World + - run: pnpm exec nx affected --base=$NX_BASE --head=$NX_HEAD -t lint test build workflows: version: 2 @@ -195,198 +119,106 @@ workflows: - main ``` -The `restore_cache` and `save_cache` steps are using a hash key that is created from the contents of the `pnpm-lock.yaml` file. This way if the `pnpm-lock.yaml` file remains the same, the next CI pipeline can pull from the cache instead of downloading `node_modules` again. This is similar to the way [Nx hashes input files to cache the results of tasks](/features/cache-task-results). - -Create a new branch with these changes and submit a PR to your repo to test them. Merge your PR into the `main` branch when you're ready to move to the next section. - -## Process Only Affected Projects +The [`nx affected` command](/ci/features/affected) will run the specified tasks only for projects that have been affected by a particular PR, which can save a lot of time as repositories grow larger. -So far we only ran the build for our `cart` application. There are other apps in our monorepo workspace though, namely `admin`, `landing-page` and `products`. We could now adjust our CI pipeline to add these builds as well: +Commit your changes and push your branch: -```plaintext -pnpm nx build cart -pnpm nx build admin -pnpm nx build landing-page +```shell +git add . +git commit -am "basic ci workflow" +git push -u origin HEAD ``` -Clearly this is not a scalable solution as it requires us to manually add every new app to the pipeline (and it doesn't include other tasks like `lint`, `test` etc). To improve this we can change the command to run the `build` for all projects like +Open up a new PR to see the run on CircleCI. If you see a message about the `nrwl/nx` orb not being loaded, you need to enable third-party CircleCI orbs in your organization settings. In the Circle CI project dashboard, go to `Organization Settings -> Security` and select `Yes` under Orb Security Settings: Allow Uncertified Orbs. -```{% command="nx run-many -t build" %} -✔ nx run shared-product-types:build (429ms) -✔ nx run shared-product-ui:build (455ms) -✔ nx run shared-header:build (467ms) -✔ nx run landing-page:build:production (3s) -✔ nx run admin:build:production (3s) -✔ nx run cart:build:production (3s) +![](/nx-cloud/tutorial/circle-orb-security.png) -———————————————————————————————————————————————————————————————— - -NX Successfully ran target build for 6 projects (10s) -``` +{% callout type="warning" title="Create Your PR on Your Own Repository" %} +Make sure that the PR you create is against your own repository's `main` branch - not the `nrwl/nx-shops` repository. +{% /callout %} -This change makes our CI pipeline configuration more maintainable. For a small repository, this might be good enough, but after a little bit of growth this approach will cause your CI times to become unmanageable. +![](/nx-cloud/tutorial/circle-new-run.avif) -Nx comes with a dedicated ["affected" command](/ci/features/affected) to help with that by only running tasks for projects that were affected by the changes in a given PR. +Once CI is green, merge the PR. -```{% command="nx affected -t build" %} -✔ nx run shared-product-types:build (404ms) -✔ nx run shared-product-ui:build (445ms) -✔ nx run shared-header:build (465ms) -✔ nx run cart:build:production (3s) +![](/nx-cloud/tutorial/github-pr-workflow.avif) -—————————————————————————————————————————————————————————————————————————————————————— +And make sure to pull the changes locally: -NX Successfully ran target build for project cart and 3 tasks it depends on (4s) +```shell +git checkout main +git pull origin main ``` -### Configuring the Comparison Range for Affected Commands - -To understand which projects are affected, Nx uses the Git history and the [project graph](/features/explore-graph). Git knows which files changed, and the Nx project graph knows which projects those files belong to. - -The affected command takes a `base` and `head` commit. The default `base` is your `main` branch and the default `head` is your current file system. This is generally what you want when developing locally, but in CI, you need to customize these values. - -The goal of the CI pipeline is to make sure that the current state of the repository is a good one. To ensure this, we want to verify all the changes **since the last successful CI run** - not just since the last commit on `main`. - -While you could calculate this yourself, we created the [`nrwl/nx` Circle CI orb](https://github.com/nrwl/nx-orb#background) to help with that. It provides you with the `nx/set-shas` step which automatically sets the `$NX_BASE` and `$NX_HEAD` environment variables to the correct commit SHAs for you to use in the affected command. - -In order to use the `nrwl/nx` orb, you need to enable the use of third-party Circle CI orbs in your organization settings. In the Circle CI project dashboard, go to `Organization Settings -> Security` and select `Yes` under Orb Security Settings: Allow Uncertified Orbs. - -![Adjust ORB Security Settings in Circle CI](/nx-cloud/tutorial/circle-orb-security.png) - -### Using the Affected Commands in our Pipeline - -Let's adjust our CI pipeline configuration to use the affected command. Create a new branch called `ci-affected` and create a PR with the following configuration: - -```yaml {% fileName=".circleci/config.yml" highlightLines=[2,3,20,22,23] %} -version: 2.1 -orbs: - nx: nrwl/nx@1.5.1 -jobs: - main: - docker: - - image: cimg/node:lts-browsers - steps: - - checkout - - restore_cache: - key: npm-dependencies-{{ checksum "pnpm-lock.yaml" }} - - run: - name: install dependencies - command: pnpm install --frozen-lockfile - - save_cache: - key: npm-dependencies-{{ checksum "pnpm-lock.yaml" }} - paths: - - node_modules - - ~/.cache/Cypress - - nx/set-shas - - - run: pnpm nx affected --base=$NX_BASE --head=$NX_HEAD -t lint test build --parallel=3 - - run: pnpm nx affected --base=$NX_BASE --head=$NX_HEAD -t e2e-ci --parallel=1 -workflows: - build: - jobs: - - main -``` +The rest of the tutorial covers remote caching and distribution across multiple machines, which need Nx Cloud to be enabled. Let's set that up next. -Notice how we're using the `$NX_BASE` and `$NX_HEAD` environment variables to set the correct `base` and `head` commits to use for file comparisons. +## Connect to Nx Cloud -We're also using the `--parallel` flag to run up to 3 `lint`, `test` or `build` tasks at once, but we want to make sure that only 1 `e2e` task is running at a time. +Nx Cloud is a companion app for your CI system that provides remote caching, task distribution, e2e test deflaking, better DX and more. -When you check the CI logs for this PR, you'll notice that no tasks were run by the `affected` command. That's because the `.circleci/config.yml` file is not an input for any task. We should really double check every task whenever we make changes to the CI pipeline, so let's fix that by adding an entry in the `sharedGlobals` array in the `nx.json` file. +Let's connect your repository to Nx Cloud with the following command: -```jsonc {% fileName="nx.json" %} -{ - "namedInputs": { - "default": ["{projectRoot}/**/*", "sharedGlobals"], - "sharedGlobals": [ - "{workspaceRoot}/babel.config.json", - "{workspaceRoot}/.circleci/config.yml" // add this line - ] - // etc... - } -} +```shell +pnpm nx connect ``` -Merge your PR into the `main` branch when you're ready to move to the next section. - -## Enable Remote Caching And Distributed Task Execution Using Nx Cloud - -Only running necessary tasks via [affected commands](/ci/features/affected) (as seen in the previous section) is helpful, but might not be enough. By default [Nx caches the results of tasks](/features/cache-task-results) on your local machine. But CI and other developer machines will still perform the same tasks on the same code - wasting time and money. Also, as your repository grows, running all the tasks on a single agent will cause the CI pipeline to take too long. The only way to decrease the CI pipeline time is to distribute your CI across many machines. Let's solve both of these problems using Nx Cloud. - -### Connect Your Workspace to Nx Cloud - -Create an account on [nx.app](https://nx.app). There are several ways to connect your repository to Nx Cloud. - -#### Connect Directly Through GitHub - -The easiest way is to create an Nx Cloud organization based on your GitHub organization. - -![Connect Your VCS Account](/nx-cloud/tutorial/connect-vcs-account.png) - -After that, connect you repository. +A browser window will open to register your repository in your [Nx Cloud](https://cloud.nx.app) account. The link is also printed to the terminal if the windows does not open, or you closed it before finishing the steps. The app will guide you to create a PR to enable Nx Cloud on your repository. -![Connect Your Repository](/nx-cloud/tutorial/connect-repository.png) +![](/nx-cloud/tutorial/nx-cloud-setup.avif) -This will send a pull request to your repository that will add the `nxCloudAccessToken` property to `nx.json`. +Nx Cloud will create a comment on your PR that gives you a summary of the CI run and a link to dig into logs and understand everything that happened during the CI run. -![Nx Cloud Setup PR](/nx-cloud/tutorial/nx-cloud-setup-pr.png) +![Nx Cloud report comment](/nx-cloud/tutorial/nx-cloud-report-comment.png) -This wires up all the CI for you and configures access. Folks who can see your repository can see your workspace on nx.app. +Once the PR is green, merge it into your main branch. -#### Manually Connect Your Workspace +![](/nx-cloud/tutorial/github-cloud-pr.avif) -To manually connect your workspace to Nx Cloud, run the following command in your repository: +And make sure you pull the latest changes locally: -```{% command="pnpm nx connect" %} -$ nx g nx:connect-to-nx-cloud --quiet --no-interactive - -NX Your Nx Cloud workspace is public - -To restrict access, connect it to your Nx Cloud account: -- Push your changes -- Login at https://cloud.nx.app to connect your repository +```shell +git pull ``` -Click the link in the terminal to claim your workspace on [nx.app](https://nx.app). - -The command generates an `nxCloudAccessToken` property inside of `nx.json`. This is a read-only token that should be committed to the repository. +You should now have an `nxCloudAccessToken` property specified in the `nx.json` file. -### Enable Remote Caching using Nx Replay +## Understand Remote Caching [Nx Cloud](https://nx.app) provides [Nx Replay](/ci/features/remote-cache), which is a powerful, scalable and, very importantly, secure way to share task artifacts across machines. It lets you configure permissions and guarantees the cached artifacts cannot be tempered with. -[Nx Replay](/ci/features/remote-cache) is enabled by default. To see it in action, rerun the CI for the PR opened by Nx Cloud. +[Nx Replay](/ci/features/remote-cache) is enabled by default. We can see it in action by running a few commands locally. First, let's build every project in the repository: -When Circle CI now processes our tasks they'll only take a fraction of the usual time. If you inspect the logs a little closer you'll see a note saying `[remote cache]`, indicating that the output has been pulled from the remote cache rather than running it. The full log of each command will still be printed since Nx restores that from the cache as well. - -![Circle CI after enabling remote caching](/nx-cloud/tutorial/circle-ci-remote-cache.png) - -![Run Details with remote cache hits](/nx-cloud/tutorial/nx-cloud-run-details.png) - -What is more, if you run tasks locally, you will also get cache hits: +```shell +pnpm nx run-many -t build +``` -```{% command="nx run-many -t build" %} -... -✔ nx run express-legacy:build [remote cache] -✔ nx run nx-plugin-legacy:build [remote cache] -✔ nx run esbuild-legacy:build [remote cache] -✔ nx run react-native-legacy:build [remote cache] -✔ nx run angular-legacy:build [remote cache] -✔ nx run remix-legacy:build [remote cache] +Nx will store the output of those tasks locally in the `.nx/cache` folder and remotely in Nx Cloud. If someone else in the organization were to run the same `build` command on the same source code, they would receive the remotely cached outputs instead of re-running the `build` task themselves. We can simulate this by deleting the `.nx/cache` folder and re-running the `build` command. -———————————————————————————————————————————————— +```shell +rm -rf .nx/cache +pnpm nx run-many -t build +``` -NX Successfully ran target build for 58 projects and 62 tasks they depend on (1m) +The `build` tasks complete almost instantly, and you can see in the logs that Nx has pulled the outputs from the remote cache: -Nx read the output from the cache instead of running the command for 116 out of 120 tasks. +``` +❯ nx run-many -t build + + ✔ nx run shared-product-types:build [remote cache] + ✔ nx run shared-product-ui:build [remote cache] + ✔ nx run shared-header:build [remote cache] + ✔ nx run landing-page:build:production [remote cache] + ✔ nx run admin:build:production [remote cache] + ✔ nx run cart:build:production [remote cache] ``` -You might also want to learn more about [how to fine-tune caching](/recipes/running-tasks/configure-inputs) to get even better results. +This remote cache is useful to speed up tasks when developing on a local machine, but it is incredibly useful for CI to be able share task results across different CI pipeline executions. When a small commit is added to a large PR, the CI is able to download the results for most of the tasks instead of recomputing everything from scratch. -Merge your PR into the `main` branch when you're ready to move to the next section. +You might also want to learn more about [how to fine-tune caching](/recipes/running-tasks/configure-inputs) to get even better results. -### Parallelize Tasks Across Multiple Machines Using Nx Agents +## Parallelize Tasks Across Multiple Machines Using Nx Agents -The affected command and Nx Replay help speed up the average CI time, but there will be some PRs that affect everything in the repository. The only way to speed up that worst case scenario is through efficient parallelization. The best way to parallelize CI with Nx is to use Nx Agents. +The affected command and Nx Replay help speed up the average CI time, but there will be some PRs that affect everything in the repository. The only way to speed up that worst case scenario is through efficient parallelization. The best way to parallelize CI with Nx is to use [Nx Agents](/ci/features/distribute-task-execution). The Nx Agents feature @@ -396,61 +228,27 @@ The Nx Agents feature - collects the results and logs of all the tasks and presents them in a single view - automatically shuts down agents when they are no longer needed -Let's enable Nx Agents +To enable Nx Agents, make sure the following line is uncommented in the `.circleci/config.yml` file. -``` -pnpm exec nx-cloud start-ci-run --distribute-on="3 linux-medium-js" --stop-agents-after="e2e-ci" +```yml {% fileName=".circleci/config.yml" highlightLines=[3] %} +# Connect your workspace on nx.app and uncomment this to enable task distribution. +# The "--stop-agents-after" is optional, but allows idle agents to shut down once the "e2e-ci" targets have been requested +- run: pnpm dlx nx-cloud start-ci-run --distribute-on="5 linux-medium-js" --stop-agents-after="e2e-ci" ``` We recommend you add this line right after you check out the repo, before installing node modules. -- `nx-cloud start-ci-run --distribute-on="3 linux-medium-js` lets Nx know that all the tasks after this line should using Nx Agents and that Nx Cloud should use 3 instances of the `linux-medium-js` launch template. See below on how to configure a custom launch template. +- `nx-cloud start-ci-run --distribute-on="5 linux-medium-js` lets Nx know that all the tasks after this line should using Nx Agents and that Nx Cloud should use 5 instances of the `linux-medium-js` launch template. See the separate reference on how to [configure a custom launch template](/ci/reference/launch-templates). - `--stop-agents-after="e2e-ci"` lets Nx Cloud know which line is the last command in this pipeline. Once there are no more e2e tasks for an agent to run, Nx Cloud will automatically shut them down. This way you're not wasting money on idle agents while a particularly long e2e task is running on a single agent. Try it out by creating a new PR with the above changes. -Once Circle CI starts, you should see multiple agents running in parallel similar to this: +Once Circle CI starts, you can click on the Nx Cloud report to see what tasks agents are executing in real time. -![CIPE Agents In Progress](/nx-cloud/tutorial/cipe-agents-in-progress.png) +![Circle CI showing multiple DTE agents](/nx-cloud/tutorial/nx-cloud-agents-in-progress.png) With this pipeline configuration in place, no matter how large the repository scales, Nx Cloud will adjust and distribute tasks across agents in the optimal way. If CI pipelines start to slow down, just add some agents. One of the main advantages is that this pipeline definition is declarative. We tell Nx what commands to run, but not how to distribute them. That way even if our monorepo structure changes and evolves over time, the distribution will be taken care of by Nx Cloud. -### Running Commands Without Distribution - -Sometimes you want to distribute most of your commands, but run some of them in Circle CI. You can do this with the `--no-agents` flag as follows: - -```yaml {% fileName=".circleci/config.yml" highlightLines=[25] %} -version: 2.1 -orbs: - nx: nrwl/nx@1.5.1 -jobs: - main: - docker: - - image: cimg/node:lts-browsers - steps: - - checkout - - run: pnpm exec nx-cloud start-ci-run --distribute-on="3 linux-medium-js" --stop-agents-after="e2e-ci" - - restore_cache: - key: npm-dependencies-{{ checksum "pnpm-lock.yaml" }} - - run: - name: install dependencies - command: pnpm install --frozen-lockfile - - save_cache: - key: npm-dependencies-{{ checksum "pnpm-lock.yaml" }} - paths: - - node_modules - - ~/.cache/Cypress - - nx/set-shas - - - run: pnpm nx affected --base=$NX_BASE --head=$NX_HEAD -t lint test build --parallel=3 - - run: pnpm nx affected --base=$NX_BASE --head=$NX_HEAD -t e2e-ci --parallel=1 - - run: pnpm nx affected --base=$NX_BASE --head=$NX_HEAD -t deploy --no-agents # run without distribution -workflows: - build: - jobs: - - main -``` - ## Next Steps You now have a highly optimized CI configuration that will scale as your repository scales. See what else you can do with Nx Cloud. diff --git a/docs/nx-cloud/tutorial/github-actions.md b/docs/nx-cloud/tutorial/github-actions.md index 382a761c134cd..3d8b2646e59d9 100644 --- a/docs/nx-cloud/tutorial/github-actions.md +++ b/docs/nx-cloud/tutorial/github-actions.md @@ -5,14 +5,12 @@ description: In this tutorial you'll set up continuous integration with GitHub A # GitHub Actions with Nx -In this tutorial we're going to learn how to leverage Nx to setup a scalable CI pipeline on GitHub Actions. You're going to learn +In this tutorial we're going to learn how to leverage Nx to setup a scalable CI pipeline on GitHub Actions. As repositories get bigger, making sure that the CI is fast, reliable and maintainable can get very challenging. Nx provides a solution. -- how to set up GitHub Actions and configure Nx -- how to run tasks for only the projects that were affected by a given PR -- how to enable remote caching -- how to parallelize and distribute tasks across multiple machines - -Note, many of these optimizations are incremental, meaning you could set up running tasks for only affected projects and stop there. Later when you experience slow CI runs, you could add caching to further improve CI performance or even go further and distribute tasks across machines. +- Nx reduces wasted time in CI with the [`affected` command](/ci/features/affected). +- Nx Replay's [remote caching](/ci/features/remote-cache) will reuse task artifacts from different CI executions making sure you will never run the same computation twice. +- Nx Agents [efficiently distribute tasks across machines](/ci/concepts/parallelization-distribution) ensuring constant CI time regardless of the repository size. The right number of machines is allocated for each PR to ensure good performance without wasting compute. +- Nx Atomizer [automatically splits](/ci/features/split-e2e-tasks) large e2e tests to distribute them across machines. Nx can also automatically [identify and rerun flaky e2e tests](/ci/features/flaky-tasks). ## Example Repository @@ -31,7 +29,7 @@ To get started: 1. [Fork the nx-shop repo](https://github.com/nrwl/nx-shops/fork) and then clone it to your local machine ```shell - git clone git@github.com:/nx-shops.git + git clone https://github.com//nx-shops.git ``` 2. Install dependencies (this repo uses [PNPM](https://pnpm.io/) but you should be able to also use any other package manager) @@ -40,87 +38,33 @@ To get started: pnpm i ``` -3. Explore the structure of the repo using **the Nx Graph** - - ```shell - pnpm nx graph - ``` - -4. Finally, make sure all task are working on your machine, by running lint, test, build and e2e on all projects of the workspace +3. Make sure all tasks are working on your machine, by running lint, test, build and e2e on all projects of the workspace ```shell - pnpm nx run-many -t lint test build e2e + pnpm nx run-many -t lint test build ``` -## Set Up GitHub Actions - -To get started with GitHub Actions, we'll create a pipeline that just logs a message. First, checkout a new branch: - -```shell -git checkout -b ci-message -``` - -Then create (or modify) the `.github/workflows/ci.yml` file with these contents: - -```yaml {% fileName=".github/workflows/ci.yml" %} -name: CI -on: - push: - branches: - # Change this if your primary branch is not main - - main - pull_request: - -jobs: - main: - runs-on: ubuntu-latest - steps: - - run: echo "Hello GitHub Actions!" -``` +## Create a CI Workflow -Next, commit this change, push the branch and create a PR on your forked GitHub repository: +First, we'll create a new branch to start adding a CI workflow. ```shell -git commit -am "pipeline that logs a message" -git push -u origin HEAD +git checkout -b setup-ci ``` -If everything was set up correctly, you should see a message from GitHub Actions in the PR with a success status. - -![All checks have passed in the PR](/nx-cloud/tutorial/gh-pr-passed.png) - -Click on the job details and you should see the `Hello GitHub Actions` message in the logs. - -![The "Hello GitHub Actions" message is printed in the logs](/nx-cloud/tutorial/gh-message.png) - -Merge your PR into the `main` branch when you're ready to move to the next section. - -## Configure Nx on GitHub Actions - -Now let's use Nx in the pipeline. The simplest way to use Nx is to run a single task, so we'll start by building our `cart` application. +Now we can use an Nx generator to create a default CI workflow file. ```shell -pnpm nx build cart +pnpm nx generate ci-workflow --ci=github ``` -We need to adjust a couple of things on our CI pipeline to make this work: - -- clone the repository -- install NPM dependencies (in our nx-shop using PNPM) -- use Nx to run the `build` command +This generator creates a `.github/workflows/ci.yml` file that contains a CI pipeline that will run the `lint`, `test`, `build` and `e2e` tasks for projects that are affected by any given PR. -Nx is an [npm package](https://www.npmjs.com/package/nx) so once NPM packages are installed we will be able to use it. +The key lines in the CI pipeline are highlighted in this excerpt: -Create a new branch called `build-one-app` and paste this code into the GitHub Actions config. - -```yaml {% fileName=".github/workflows/ci.yml" highlightLines=["12-20"] %} +```yml {% fileName=".github/workflows/ci.yml" highlightLines=["31-33"] %} name: CI -on: - push: - branches: - - main - pull_request: - +# ... jobs: main: runs-on: ubuntu-latest @@ -128,269 +72,120 @@ jobs: - uses: actions/checkout@v4 with: fetch-depth: 0 - # Setup pnpm - - uses: pnpm/action-setup@v2 - with: - version: 8 - - run: pnpm install --frozen-lockfile - - run: pnpm nx build cart -``` - -Once `node_modules` are in place, you can run normal Nx commands. In this case, we run `pnpm nx build cart`. Push the changes to your repository by creating a new PR and verifying the new CI pipeline correctly builds our application. - -![Building a single app with nx](/nx-cloud/tutorial/gh-single-build-success.png) - -You might have noticed that there's also a build running for `shared-header`, `shared-product-types` and `shared-product-ui`. These are projects in our workspace that `cart` depends on. Thanks to the [Nx task pipeline](/concepts/task-pipeline-configuration), Nx knows that it needs to build these projects first before building `cart`. This already helps us simplify our pipeline as we - -- don't need to define these builds automatically -- don't need to make any changes to our pipeline as our `cart` app grows and depends on more projects -- don't need to worry about the order of the builds - -Merge your PR into the `main` branch when you're ready to move to the next section. - -## Optimize our CI by caching NPM dependencies - -While this isn't related to Nx specifically, it's a good idea to cache NPM dependencies in CI. This will speed up the CI pipeline by avoiding downloading the same dependencies over and over again. GitHub Actions has [an action to cache files](https://github.com/actions/cache) that we'll use. -Adjust your CI pipeline script as follows - -```yaml {% fileName=".github/workflows/ci.yml" highlightLines=["18-24", "26-32"] %} -name: CI -on: - push: - branches: - - main - pull_request: - -jobs: - main: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - with: - fetch-depth: 0 - uses: pnpm/action-setup@v2 with: version: 8 - - name: Restore cached npm dependencies - uses: actions/cache/restore@v3 - with: - path: | - node_modules - ~/.cache/Cypress # needed for the Cypress binary - key: npm-dependencies-${{ hashFiles('pnpm-lock.yaml') }} - - run: pnpm install --frozen-lockfile - - name: Cache npm dependencies - uses: actions/cache/save@v3 - with: - path: | - node_modules - ~/.cache/Cypress # needed for the Cypress binary - key: npm-dependencies-${{ hashFiles('pnpm-lock.yaml') }} - - run: pnpm nx build cart -``` - -The `restore_cache` and `save_cache` steps are using a hash key that is created from the contents of the `pnpm-lock.yaml` file. This way if the `pnpm-lock.yaml` file remains the same, the next CI pipeline can pull from the cache instead of downloading `node_modules` again. This is similar to the way [Nx hashes input files to cache the results of tasks](/features/cache-task-results). - -Create a new branch with these changes and submit a PR to your repo to test them. Merge your PR into the `main` branch when you're ready to move to the next section. -## Process Only Affected Projects + # This enables task distribution via Nx Cloud + # Run this command as early as possible, before dependencies are installed + # Learn more at https://nx.dev/ci/reference/nx-cloud-cli#npx-nxcloud-startcirun + # Connect your workspace by running "nx connect" and uncomment this + # - run: pnpm dlx nx-cloud start-ci-run --distribute-on="3 linux-medium-js" --stop-agents-after="e2e-ci" -So far we only ran the build for our `cart` application. There are other apps in our monorepo workspace though, namely `admin`, `landing-page` and `products`. We could now adjust our CI pipeline to add these builds as well: - -```plaintext -pnpm nx build cart -pnpm nx build admin -pnpm nx build landing-page -``` - -Clearly this is not a scalable solution as it requires us to manually add every new app to the pipeline (and it doesn't include other tasks like `lint`, `test` etc). To improve this we can change the command to run the `build` for all projects like - -```{% command="nx run-many -t build" %} -✔ nx run shared-product-types:build (429ms) -✔ nx run shared-product-ui:build (455ms) -✔ nx run shared-header:build (467ms) -✔ nx run landing-page:build:production (3s) -✔ nx run admin:build:production (3s) -✔ nx run cart:build:production (3s) + - uses: actions/setup-node@v3 + with: + node-version: 20 + cache: 'pnpm' -———————————————————————————————————————————————————————————————— + - run: pnpm install --frozen-lockfile + - uses: nrwl/nx-set-shas@v4 -NX Successfully ran target build for 6 projects (10s) + # Prepend any command with "nx-cloud record --" to record its logs to Nx Cloud + # - run: pnpm exec nx-cloud record -- echo Hello World + # Nx Affected runs only tasks affected by the changes in this PR/commit. Learn more: https://nx.dev/ci/features/affected + - run: pnpm exec nx affected -t lint test build ``` -This change makes our CI pipeline configuration more maintainable. For a small repository, this might be good enough, but after a little bit of growth this approach will cause your CI times to become unmanageable. - -Nx comes with a dedicated ["affected" command](/ci/features/affected) to help with that by only running tasks for projects that were affected by the changes in a given PR. - -```{% command="nx affected -t build" %} -✔ nx run shared-product-types:build (404ms) -✔ nx run shared-product-ui:build (445ms) -✔ nx run shared-header:build (465ms) -✔ nx run cart:build:production (3s) +The [`nx affected` command](/ci/features/affected) will run the specified tasks only for projects that have been affected by a particular PR, which can save a lot of time as repositories grow larger. -—————————————————————————————————————————————————————————————————————————————————————— +Commit your changes and push your branch: -NX Successfully ran target build for project cart and 3 tasks it depends on (4s) +```shell +git add . +git commit -am "basic ci workflow" +git push -u origin HEAD ``` -### Configuring the Comparison Range for Affected Commands +Create a pull request with the new branch and watch your CI in action. -To understand which projects are affected, Nx uses the Git history and the [project graph](/features/explore-graph). Git knows which files changed, and the Nx project graph knows which projects those files belong to. +{% callout type="warning" title="Create Your PR on Your Own Repository" %} +Make sure that the PR you create is against your own repository's `main` branch - not the `nrwl/nx-shops` repository. +{% /callout %} -The affected command takes a `base` and `head` commit. The default `base` is your `main` branch and the default `head` is your current file system. This is generally what you want when developing locally, but in CI, you need to customize these values. +Once CI is green, merge the PR. -The goal of the CI pipeline is to make sure that the current state of the repository is a good one. To ensure this, we want to verify all the changes **since the last successful CI run** - not just since the last commit on `main`. +![](/nx-cloud/tutorial/github-pr-workflow.avif) -While you could calculate this yourself, we created the [`nx-set-shas` GitHub Action](https://github.com/marketplace/actions/nx-set-shas) to help with that. It provides you with the `nrwl/nx-set-shas` action which automatically sets the `NX_BASE` and `NX_HEAD` environment variables to the correct commit SHAs. The affected command will use these environment variables when they are defined. +The rest of the tutorial covers remote caching and distribution across multiple machines, which need Nx Cloud to be enabled. Let's set that up next. -### Using the Affected Commands in our Pipeline +## Connect to Nx Cloud -Let's adjust our CI pipeline configuration to use the affected command. Create a new branch called `ci-affected` and create a PR with the following configuration: - -```yaml {% fileName=".github/workflows/ci.yml" highlightLines=["35-39"] %} -name: CI -on: - push: - branches: - - main - pull_request: +Nx Cloud is a companion app for your CI system that provides remote caching, task distribution, e2e test deflaking, better DX and more. -jobs: - main: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - with: - fetch-depth: 0 - - uses: pnpm/action-setup@v2 - with: - version: 8 - - name: Restore cached npm dependencies - id: cache-dependencies-restore - uses: actions/cache/restore@v3 - with: - path: | - node_modules - ~/.cache/Cypress # needed for the Cypress binary - key: npm-dependencies-${{ hashFiles('pnpm-lock.yaml') }} - - run: pnpm install --frozen-lockfile - - name: Cache npm dependencies - id: cache-dependencies-save - uses: actions/cache/save@v3 - with: - path: | - node_modules - ~/.cache/Cypress # needed for the Cypress binary - key: ${{ steps.cache-dependencies-restore.outputs.cache-primary-key }} - - uses: nrwl/nx-set-shas@v3 - # This line is needed for nx affected to work when CI is running on a PR - - run: git branch --track main origin/main - if: ${{ github.event_name == 'pull_request' }} - - - run: pnpm nx affected -t lint test build --parallel=3 - - run: pnpm nx affected -t e2e --parallel=1 -``` +Let's connect your repository to Nx Cloud with the following command: -We're using the `--parallel` flag to run up to 3 `lint`, `test` or `build` tasks at once, but we want to make sure that only 1 `e2e` task is running at a time. - -When you check the CI logs for this PR, you'll notice that no tasks were run by the `affected` command. That's because the `.github/workflows/ci.yml` file is not an input for any task. We should really double check every task whenever we make changes to the CI pipeline, so let's fix that by adding an entry in the `sharedGlobals` array in the `nx.json` file. - -```jsonc {% fileName="nx.json" highlightLines=[6] %} -{ - "namedInputs": { - "default": ["{projectRoot}/**/*", "sharedGlobals"], - "sharedGlobals": [ - "{workspaceRoot}/babel.config.json", - "{workspaceRoot}/.github/workflows/ci.yml" // add this line - ] - // etc... - } -} +```shell +pnpm nx connect ``` -Merge your PR into the `main` branch when you're ready to move to the next section. - -## Enable Remote Caching and Distributed Task Execution Using Nx Cloud +A browser window will open to register your repository in your [Nx Cloud](https://cloud.nx.app) account. The link is also printed to the terminal if the windows does not open, or you closed it before finishing the steps. The app will guide you to create a PR to enable Nx Cloud on your repository. -Only running necessary tasks via [affected commands](/ci/features/affected) (as seen in the previous section) is helpful, but might not be enough. By default [Nx caches the results of tasks](/features/cache-task-results) on your local machine. But CI and other developer machines will still perform the same tasks on the same code - wasting time and money. Also, as your repository grows, running all the tasks on a single agent will cause the CI pipeline to take too long. The only way to decrease the CI pipeline time is to distribute your CI across many machines. Let's solve both of these problems using Nx Cloud. +![](/nx-cloud/tutorial/nx-cloud-setup.avif) -### Connect Your Workspace to Nx Cloud +Nx Cloud will create a comment on your PR that gives you a summary of the CI run and a link to dig into logs and understand everything that happened during the CI run. -Create an account on [nx.app](https://nx.app). The easiest way to connect your repository to Nx Cloud is to create an Nx Cloud organization based on your GitHub organization. +![Nx Cloud report comment](/nx-cloud/tutorial/nx-cloud-report-comment.png) -![Connect Your VCS Account](/nx-cloud/tutorial/connect-vcs-account.png) +Once the PR is green, merge it into your main branch. -After that, connect you repository. +![](/nx-cloud/tutorial/github-cloud-pr.avif) -![Connect Your Repository](/nx-cloud/tutorial/connect-repository.png) +And make sure you pull the latest changes locally: -This will send a pull request to your repository that will add the `nxCloudAccessToken` property to `nx.json`. - -![Nx Cloud Setup PR](/nx-cloud/tutorial/nx-cloud-setup-pr.png) +```shell +git pull +``` -This wires up all the CI for you and configures access. Folks who can see your repository can see your workspace on nx.app. +You should now have an `nxCloudAccessToken` property specified in the `nx.json` file. -### Enable Remote Caching using Nx Replay +## Understand Remote Caching [Nx Cloud](https://nx.app) provides [Nx Replay](/ci/features/remote-cache), which is a powerful, scalable and, very importantly, secure way to share task artifacts across machines. It lets you configure permissions and guarantees the cached artifacts cannot be tempered with. -[Nx Replay](/ci/features/remote-cache) is enabled by default. To see it in action, rerun the CI for the PR opened by Nx Cloud. - -When GitHub Actions now processes our tasks they'll only take a fraction of the usual time. If you inspect the logs a little closer you'll see a note saying `[remote cache]`, indicating that the output has been pulled from the remote cache rather than running it. The full log of each command will still be printed since Nx restores that from the cache as well. +[Nx Replay](/ci/features/remote-cache) is enabled by default. We can see it in action by running a few commands locally. First, let's build every project in the repository: -![GitHub Actions after enabling remote caching](/nx-cloud/tutorial/gh-ci-remote-cache.png) - -![Run Details with remote cache hits](/nx-cloud/tutorial/nx-cloud-run-details.png) - -What is more, if you run tasks locally, you will also get cache hits: - -```{% command="nx run-many -t build" %} -... -✔ nx run express-legacy:build [remote cache] -✔ nx run nx-plugin-legacy:build [remote cache] -✔ nx run esbuild-legacy:build [remote cache] -✔ nx run react-native-legacy:build [remote cache] -✔ nx run angular-legacy:build [remote cache] -✔ nx run remix-legacy:build [remote cache] - -———————————————————————————————————————————————— - -NX Successfully ran target build for 58 projects and 62 tasks they depend on (1m) - -Nx read the output from the cache instead of running the command for 116 out of 120 tasks. +```shell +pnpm nx run-many -t build ``` -You might also want to learn more about [how to fine-tune caching](/recipes/running-tasks/configure-inputs) to get even better results. - -Merge your PR into the `main` branch when you're ready to move to the next section. - -## Enable PR Integration - -The [Nx Cloud GitHub App](https://github.com/marketplace/official-nx-cloud-app) automatically creates a comment on your PRs that provides a direct link to the relevant Nx Cloud logs and quickly shows which command failed. - -### Install the App +Nx will store the output of those tasks locally in the `.nx/cache` folder and remotely in Nx Cloud. If someone else in the organization were to run the same `build` command on the same source code, they would receive the remotely cached outputs instead of re-running the `build` task themselves. We can simulate this by deleting the `.nx/cache` folder and re-running the `build` command. -Install the [Nx Cloud GitHub App](https://github.com/marketplace/official-nx-cloud-app) and give it permission to access your repo. - -### Connecting Your Workspace - -Once you have installed the Nx Cloud GitHub App, you must link your workspace to the installation. To do this, sign in to Nx Cloud and navigate to the VCS Integrations setup page. Once on the VCS Integrations setup page, choose GitHub as your version control system. - -![Access VCS Setup](/nx-cloud/set-up/access-vcs-setup.webp) +```shell +rm -rf .nx/cache +pnpm nx run-many -t build +``` -### Authenticate Via the GitHub App +The `build` tasks complete almost instantly, and you can see in the logs that Nx has pulled the outputs from the remote cache: -To use the Nx Cloud GitHub App for authentication, select the radio button and then click "Connect". -This will verify that Nx Cloud can connect to your repo. Upon a successful test, your configuration is saved. +``` +❯ nx run-many -t build + + ✔ nx run shared-product-types:build [remote cache] + ✔ nx run shared-product-ui:build [remote cache] + ✔ nx run shared-header:build [remote cache] + ✔ nx run landing-page:build:production [remote cache] + ✔ nx run admin:build:production [remote cache] + ✔ nx run cart:build:production [remote cache] +``` -![Use GitHub App for Authentication](/nx-cloud/set-up/use-github-app-auth.webp) +This remote cache is useful to speed up tasks when developing on a local machine, but it is incredibly useful for CI to be able share task results across different CI pipeline executions. When a small commit is added to a large PR, the CI is able to download the results for most of the tasks instead of recomputing everything from scratch. -Now any new PRs in your repo should have a comment automatically added that links directly to Nx Cloud. For other ways of setting up PR integration, read the [Enable GitHub PR Integration recipe](/ci/recipes/source-control-integration/github). +You might also want to learn more about [how to fine-tune caching](/recipes/running-tasks/configure-inputs) to get even better results. -### Parallelize Tasks Across Multiple Machines Using Nx Agents +## Parallelize Tasks Across Multiple Machines Using Nx Agents -The affected command and Nx Replay help speed up the average CI time, but there will be some PRs that affect everything in the repository. The only way to speed up that worst case scenario is through efficient parallelization. The best way to parallelize CI with Nx is to use Nx Agents. +The affected command and Nx Replay help speed up the average CI time, but there will be some PRs that affect everything in the repository. The only way to speed up that worst case scenario is through efficient parallelization. The best way to parallelize CI with Nx is to use [Nx Agents](/ci/features/distribute-task-execution). The Nx Agents feature @@ -400,77 +195,27 @@ The Nx Agents feature - collects the results and logs of all the tasks and presents them in a single view - automatically shuts down agents when they are no longer needed -Let's enable Nx Agents +To enable Nx Agents, make sure the following line is uncommented in the `.github/workflows/ci.yml` file. -```shell -pnpm dlx nx-cloud start-ci-run --distribute-on="3 linux-medium-js" --stop-agents-after="e2e-ci" +```yml {% fileName=".github/workflows/ci.yml" highlightLines=[3] %} +# Connect your workspace on nx.app and uncomment this to enable task distribution. +# The "--stop-agents-after" is optional, but allows idle agents to shut down once the "e2e-ci" targets have been requested +- run: pnpm dlx nx-cloud start-ci-run --distribute-on="5 linux-medium-js" --stop-agents-after="e2e-ci" ``` We recommend you add this line right after you check out the repo, before installing node modules. -- `nx-cloud start-ci-run --distribute-on="3 linux-medium-js` lets Nx know that all the tasks after this line should using Nx Agents and that Nx Cloud should use 3 instances of the `linux-medium-js` launch template. See below on how to configure a custom launch template. +- `nx-cloud start-ci-run --distribute-on="5 linux-medium-js` lets Nx know that all the tasks after this line should using Nx Agents and that Nx Cloud should use 5 instances of the `linux-medium-js` launch template. See the separate reference on how to [configure a custom launch template](/ci/reference/launch-templates). - `--stop-agents-after="e2e-ci"` lets Nx Cloud know which line is the last command in this pipeline. Once there are no more e2e tasks for an agent to run, Nx Cloud will automatically shut them down. This way you're not wasting money on idle agents while a particularly long e2e task is running on a single agent. Try it out by creating a new PR with the above changes. -Once GitHub Actions starts, you should see multiple agents running in parallel: +Once GitHub Actions starts, you can click on the Nx Cloud report to see what tasks agents are executing in real time. -![GitHub Actions showing multiple DTE agents](/nx-cloud/tutorial/gh-dte-multiple-agents.png) +![GitHub Actions showing multiple DTE agents](/nx-cloud/tutorial/nx-cloud-agents-in-progress.png) With this pipeline configuration in place, no matter how large the repository scales, Nx Cloud will adjust and distribute tasks across agents in the optimal way. If CI pipelines start to slow down, just add some agents. One of the main advantages is that this pipeline definition is declarative. We tell Nx what commands to run, but not how to distribute them. That way even if our monorepo structure changes and evolves over time, the distribution will be taken care of by Nx Cloud. -### Running Commands Without Distribution - -Sometimes you want to distribute most of your commands, but run some of them in Github Actions. You can do this with the `--no-agents` flag as follows: - -```yaml {% fileName=".github/workflows/ci.yml" highlightLines=["18-21","44"] %} -name: CI -on: - push: - branches: - - main - pull_request: - -jobs: - main: - runs-on: ubuntu-latest - steps: - - uses: actions/checkout@v4 - with: - fetch-depth: 0 - - uses: pnpm/action-setup@v2 - with: - version: 8 - - run: | - pnpm dlx nx-cloud start-ci-run \ - --distribute-on="3 linux-medium-js" \ - --stop-agents-after="e2e-ci" - - name: Restore cached npm dependencies - id: cache-dependencies-restore - uses: actions/cache/restore@v3 - with: - path: | - node_modules - ~/.cache/Cypress # needed for the Cypress binary - key: npm-dependencies-${{ hashFiles('pnpm-lock.yaml') }} - - run: pnpm install --frozen-lockfile - - name: Cache npm dependencies - id: cache-dependencies-save - uses: actions/cache/save@v3 - with: - path: | - node_modules - ~/.cache/Cypress # needed for the Cypress binary - key: ${{ steps.cache-dependencies-restore.outputs.cache-primary-key }} - - uses: nrwl/nx-set-shas@v3 - # This line is needed for nx affected to work when CI is running on a PR - - run: git branch --track main origin/main - if: ${{ github.event_name == 'pull_request' }} - - run: pnpm nx affected -t lint test build --parallel=3 - - run: pnpm nx affected -t e2e-ci --parallel=1 - - run: pnpm nx affected -t deploy --no-agents -``` - ## Next Steps You now have a highly optimized CI configuration that will scale as your repository scales. See what else you can do with Nx Cloud. diff --git a/docs/nx-cloud/tutorial/github-cloud-pr.avif b/docs/nx-cloud/tutorial/github-cloud-pr.avif new file mode 100644 index 0000000000000..971813d6c8f0a Binary files /dev/null and b/docs/nx-cloud/tutorial/github-cloud-pr.avif differ diff --git a/docs/nx-cloud/tutorial/github-pr-workflow.avif b/docs/nx-cloud/tutorial/github-pr-workflow.avif new file mode 100644 index 0000000000000..b5c4653fb6bb3 Binary files /dev/null and b/docs/nx-cloud/tutorial/github-pr-workflow.avif differ diff --git a/docs/nx-cloud/tutorial/nx-cloud-agents-in-progress.png b/docs/nx-cloud/tutorial/nx-cloud-agents-in-progress.png new file mode 100644 index 0000000000000..a18172056e878 Binary files /dev/null and b/docs/nx-cloud/tutorial/nx-cloud-agents-in-progress.png differ diff --git a/docs/nx-cloud/tutorial/nx-cloud-report-comment.png b/docs/nx-cloud/tutorial/nx-cloud-report-comment.png new file mode 100644 index 0000000000000..26a06cf71a86e Binary files /dev/null and b/docs/nx-cloud/tutorial/nx-cloud-report-comment.png differ diff --git a/docs/nx-cloud/tutorial/nx-cloud-setup.avif b/docs/nx-cloud/tutorial/nx-cloud-setup.avif new file mode 100644 index 0000000000000..5a19fc17c6e69 Binary files /dev/null and b/docs/nx-cloud/tutorial/nx-cloud-setup.avif differ