Skip to content

Commit

Permalink
new: Allow targets to be passed to moon ci. (#1056)
Browse files Browse the repository at this point in the history
* Update ci.

* Update targets.

* Remove job.
  • Loading branch information
milesj committed Sep 25, 2023
1 parent b24b4d2 commit b8a3c7c
Show file tree
Hide file tree
Showing 5 changed files with 78 additions and 36 deletions.
57 changes: 35 additions & 22 deletions crates/cli/src/commands/ci.rs
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,9 @@ const HEADING_PARALLELISM: &str = "Parallelism and distribution";

#[derive(Args, Clone, Debug)]
pub struct CiArgs {
#[arg(help = "List of targets (scope:task) to run")]
targets: Vec<Target>,

#[arg(long, help = "Base branch, commit, or revision to compare against")]
base: Option<String>,

Expand Down Expand Up @@ -88,37 +91,35 @@ async fn gather_touched_files(
fn gather_runnable_targets(
provider: &CiOutput,
project_graph: &ProjectGraph,
touched_files: &FxHashSet<WorkspaceRelativePathBuf>,
args: &CiArgs,
) -> AppResult<TargetList> {
print_header(provider, "Gathering runnable targets");

let mut targets = vec![];

// Required for dependents
for project in project_graph.get_all()? {
for task in project.get_tasks()? {
if task.should_run_in_ci() {
if task.is_affected(touched_files)? {
let projects = project_graph.get_all()?;

if args.targets.is_empty() {
for project in projects {
for task in project.get_tasks()? {
if task.should_run_in_ci() {
targets.push(task.target.clone());
} else {
debug!(
"Not running target {} because it either has no {} or {} is false",
color::label(&task.target.id),
color::property("outputs"),
color::property("runInCI"),
);
}
} else {
debug!(
"Not running target {} because it either has no `outputs` or `runInCI` is false",
color::label(&task.target),
);
}
}
}

if targets.is_empty() {
println!(
"{}",
color::invalid("No targets to run based on touched files")
);
} else {
print_targets(&targets);
targets.extend(args.targets.clone());
}

print_targets(&targets);
print_footer(provider);

Ok(targets)
Expand Down Expand Up @@ -165,14 +166,15 @@ fn generate_dep_graph(
provider: &CiOutput,
project_graph: &ProjectGraph,
targets: &TargetList,
touched_files: &FxHashSet<WorkspaceRelativePathBuf>,
) -> AppResult<DepGraph> {
print_header(provider, "Generating dependency graph");

let mut dep_builder = build_dep_graph(project_graph);

for target in targets {
// Run the target and its dependencies
dep_builder.run_target(target, None)?;
dep_builder.run_target(target, Some(touched_files))?;

// And also run its dependents to ensure consumers still work correctly
dep_builder.run_dependents_for_target(target)?;
Expand All @@ -199,17 +201,28 @@ pub async fn ci(
});
let project_graph = generate_project_graph(workspace).await?;
let touched_files = gather_touched_files(&ci_provider, workspace, args).await?;
let targets = gather_runnable_targets(&ci_provider, &project_graph, &touched_files)?;
let targets = gather_runnable_targets(&ci_provider, &project_graph, args)?;

if targets.is_empty() {
println!("{}", color::invalid("No targets to run"));

return Ok(());
}

let targets = distribute_targets_across_jobs(&ci_provider, args, targets);
let dep_graph = generate_dep_graph(&ci_provider, &project_graph, &targets)?;
let dep_graph = generate_dep_graph(&ci_provider, &project_graph, &targets, &touched_files)?;

if dep_graph.is_empty() {
println!(
"{}",
color::invalid("No targets to run based on touched files")
);

return Ok(());
}

// Process all tasks in the graph
print_header(&ci_provider, "Running all targets");
print_header(&ci_provider, "Running targets");

let context = ActionContext {
primary_targets: FxHashSet::from_iter(targets),
Expand Down
4 changes: 4 additions & 0 deletions crates/core/dep-graph/src/dep_graph.rs
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,10 @@ impl DepGraph {
DepGraph { graph, indices }
}

pub fn is_empty(&self) -> bool {
self.get_node_count() == 0
}

pub fn get_index_from_node(&self, node: &ActionNode) -> Option<&NodeIndex> {
self.indices.get(node)
}
Expand Down
1 change: 1 addition & 0 deletions packages/cli/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
- Better concurrency handling and scheduling.
- More accurately monitors signals (ctrl+c) and shutdowns.
- Tasks can now be configured with a timeout.
- Updated `moon ci` to support running a list of targets, instead of running everything.

#### ⚙️ Internal

Expand Down
13 changes: 13 additions & 0 deletions website/docs/commands/ci.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,26 @@ title: ci
The `moon ci` command is a special command that should be ran in a continuous integration (CI)
environment, as it does all the heavy lifting necessary for effectively running tasks.

By default this will run all tasks that are affected by touched files and have the
[`runInCI`](../config/project#runinci) task option enabled.

```shell
$ moon ci
```

However, you can provide a list of targets to run, instead of relying on `runInCI`.

```shell
$ moon ci :build :lint
```

> View the official [continuous integration guide](../guides/ci) for a more in-depth example of how
> to utilize this command.
### Arguments

- `...[target]` - [Targets](../concepts/target) to run.

### Options

- `--base <rev>` - Base branch, commit, or revision to compare against. Can be set with `MOON_BASE`.
Expand Down
39 changes: 25 additions & 14 deletions website/docs/guides/ci.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ title: Continuous integration (CI)
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import Image from '@site/src/components/Image';
import VersionLabel from '@site/src/components/Docs/VersionLabel';

All companies and projects rely on continuous integration (CI) to ensure high quality code and to
avoid regressions. Because this is such a critical piece of every developer's workflow, we wanted to
Expand Down Expand Up @@ -50,15 +51,10 @@ by default. This can be easily controlled with the [`local`](../config/project#l

## Integrating

Although moon has an [integrated toolchain](../concepts/toolchain), we still require Node.js and
dependencies to be installed _beforehand_, as moon is currently shipped as an
[npm package](https://www.npmjs.com/package/@moonrepo/cli). This is unfortunate and we're looking
into other distribution channels.

With that being said, the following examples can be referenced for setting up moon and its CI
workflow in popular services. The examples assume a
[package script named `moon`](../install#adding-a-package-script) and are using Yarn 3, but feel
free to replace with your chosen setup.
The following examples can be referenced for setting up moon and its CI workflow in popular
providers. For GitHub, we're using our
[`setup-moon` action](https://github.com/moonrepo/setup-moon-action) to install moon. For other
providers, we assume moon is an npm dependency and must be installed with Node.js.

<Tabs groupId="ci-env">
<TabItem value="github" label="GitHub">
Expand All @@ -78,11 +74,8 @@ jobs:
- uses: 'actions/checkout@v4'
with:
fetch-depth: 0
- uses: 'actions/setup-node@v3'
with:
cache: 'yarn'
- run: 'yarn install --immutable'
- run: 'yarn moon ci'
- uses: 'moonrepo/setup-moon-action@v1'
- run: 'moon ci'
```

</TabItem>
Expand Down Expand Up @@ -136,6 +129,24 @@ script: 'moon ci'
</TabItem>
</Tabs>

## Choosing targets<VersionLabel version="1.14.0" />

By default `moon ci` will run _all_ tasks from _all_ projects that are affected by touched files and
have the [`runInCI`](../config/project#runinci) task option enabled. This is a great catch-all
solution, but may not vibe with your workflow or requirements.

If you'd prefer more control, you can pass a list of targets to `moon ci`, instead of moon
attempting to detect them. When providing targets, `moon ci` will still only run them if affected by
touched files, but will ignore the `runInCI` option.

```shell
# Run all builds
$ moon ci :build
# In another job, run tests
$ moon ci :test :lint
```

## Comparing revisions

By default the command will compare the current HEAD against a base revision, which is typically the
Expand Down

0 comments on commit b8a3c7c

Please sign in to comment.