Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new: Add cache base warmup strategy. #16

Merged
merged 6 commits into from
Nov 17, 2023
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,7 @@
# 1.1.0

- Updated dependencies.

# 1.0.3

- Include `GITHUB_WORKFLOW` in cache key.
Expand Down
4 changes: 4 additions & 0 deletions action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,10 @@ inputs:
cache:
description: 'Toggle caching of ~/.cargo/registry and /target/<cache-target> directories.'
default: true
cache-base:
description:
'Base branch or ref to save a warmup cache. Other branches and refs will restore from this
base.'
cache-target:
description: 'Name of the target profile to cache.'
default: 'debug'
Expand Down
3 changes: 2 additions & 1 deletion index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@ import * as core from '@actions/core';
import * as exec from '@actions/exec';
import * as io from '@actions/io';
import * as tc from '@actions/tool-cache';
import { CARGO_HOME, installBins, restoreCache } from './src/cargo';
import { CARGO_HOME } from './src/cache';
import { installBins, restoreCache } from './src/cargo';
import { installToolchain } from './src/rust';

export async function installRustup() {
Expand Down
4 changes: 2 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -26,11 +26,11 @@
"detect-libc": "^2.0.2"
},
"devDependencies": {
"@types/node": "^20.9.0",
"@types/node": "^20.9.1",
"@vercel/ncc": "^0.38.1",
"eslint": "^8.53.0",
"eslint-config-moon": "^2.0.11",
"prettier": "^3.0.3",
"prettier": "^3.1.0",
"prettier-config-moon": "^1.1.2",
"tsconfig-moon": "^1.3.0",
"typescript": "^5.2.2"
Expand Down
20 changes: 10 additions & 10 deletions pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 8 additions & 1 deletion post.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,14 @@ import { saveCache } from './src/cargo';

async function run() {
try {
await saveCache();
const base = core.getInput('cache-base');

// Only save the cache for the following 2 scenarios:
// - If not using the base warmup strategy.
// - If using the base warmup strategy, and the current ref matches.
if (!base || (base && !!(process.env.GITHUB_REF_NAME ?? '').match(base))) {
await saveCache();
}
} catch (error: unknown) {
core.setFailed((error as Error).message);
}
Expand Down
77 changes: 77 additions & 0 deletions src/cache.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
import crypto from 'node:crypto';
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

moved cache stuff to its own file

import os from 'node:os';
import path from 'node:path';
import * as cache from '@actions/cache';
import * as core from '@actions/core';
import * as glob from '@actions/glob';
import { RUST_HASH, RUST_VERSION } from './rust';

export const CARGO_HOME = process.env.CARGO_HOME ?? path.join(os.homedir(), '.cargo');

export const WORKSPACE_ROOT = process.env.GITHUB_WORKSPACE ?? process.cwd();

export function isCacheEnabled(): boolean {
return core.getBooleanInput('cache') && cache.isFeatureAvailable();
}

export function getCacheTarget(): string {
return core.getInput('cache-target') || 'debug';
}

export function getCachePaths(): string[] {
return [
// ~/.cargo/registry
path.join(CARGO_HOME, 'registry'),
// /workspace/target/debug
path.join(WORKSPACE_ROOT, 'target', getCacheTarget()),
];
}

export function getCachePrefixes(): string[] {
return [`setup-rustcargo-v1-${process.platform}`, 'setup-rustcargo-v1'];
}

export async function getPrimaryCacheKey() {
const hasher = crypto.createHash('sha1');

core.info('Generating cache key');

core.debug(`Hashing Rust version = ${RUST_VERSION}`);
hasher.update(RUST_VERSION);

core.debug(`Hashing Rust commit hash = ${RUST_HASH}`);
hasher.update(RUST_HASH);

const cacheTarget = getCacheTarget();

core.debug(`Hashing target profile = ${cacheTarget}`);
hasher.update(cacheTarget);

if (core.getInput('cache-base')) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Big change here to the cache key

core.debug('Using warmup strategy, not hashing Cargo.lock, GITHUB_WORKFLOW, or GITHUB_JOB');
}

// When warming up, these add far too much granularity to the cache key
else {
const lockHash = await glob.hashFiles('Cargo.lock');

core.debug(`Hashing Cargo.lock = ${lockHash}`);
hasher.update(lockHash);

const workflow = process.env.GITHUB_WORKFLOW;

if (workflow) {
core.debug(`Hashing GITHUB_WORKFLOW = ${workflow}`);
hasher.update(workflow);
}

const job = process.env.GITHUB_JOB;

if (job) {
core.debug(`Hashing GITHUB_JOB = ${job}`);
hasher.update(job);
}
}

return `${getCachePrefixes()[0]}-${hasher.digest('hex')}`;
}
79 changes: 12 additions & 67 deletions src/cargo.ts
Original file line number Diff line number Diff line change
@@ -1,21 +1,21 @@
import crypto from 'node:crypto';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
import { family } from 'detect-libc';
import * as cache from '@actions/cache';
import * as core from '@actions/core';
import * as exec from '@actions/exec';
import * as glob from '@actions/glob';
import * as tc from '@actions/tool-cache';
import {
CARGO_HOME,
getCachePaths,
getCachePrefixes,
getCacheTarget,
getPrimaryCacheKey,
isCacheEnabled,
WORKSPACE_ROOT,
} from './cache';
import { rmrf } from './fs';
import { RUST_HASH, RUST_VERSION } from './rust';

export const CARGO_HOME = process.env.CARGO_HOME ?? path.join(os.homedir(), '.cargo');

export const WORKSPACE_ROOT = process.env.GITHUB_WORKSPACE ?? process.cwd();

export const CACHE_ENABLED = core.getBooleanInput('cache') && cache.isFeatureAvailable();

export async function downloadAndInstallBinstall(binDir: string) {
core.info('cargo-binstall does not exist, attempting to install');
Expand Down Expand Up @@ -79,7 +79,7 @@ export async function installBins() {
.map((bin) => bin.trim())
.filter(Boolean);

if (CACHE_ENABLED) {
if (isCacheEnabled()) {
bins.push('cargo-cache');
}

Expand All @@ -98,61 +98,6 @@ export async function installBins() {
await exec.exec('cargo', ['binstall', '--no-confirm', '--log-level', 'info', ...bins]);
}

export function getCacheTarget(): string {
return core.getInput('cache-target') || 'debug';
}

export function getCachePaths(): string[] {
return [
// ~/.cargo/registry
path.join(CARGO_HOME, 'registry'),
// /workspace/target/debug
path.join(WORKSPACE_ROOT, 'target', getCacheTarget()),
];
}

export function getCachePrefixes(): string[] {
return [`setup-rustcargo-v1-${process.platform}`, 'setup-rustcargo-v1'];
}

export async function getPrimaryCacheKey() {
const hasher = crypto.createHash('sha1');

core.info('Generating cache key');

core.debug(`Hashing Rust version = ${RUST_VERSION}`);
hasher.update(RUST_VERSION);

core.debug(`Hashing Rust commit hash = ${RUST_HASH}`);
hasher.update(RUST_HASH);

const lockHash = await glob.hashFiles('Cargo.lock');

core.debug(`Hashing Cargo.lock = ${lockHash}`);
hasher.update(lockHash);

const cacheTarget = getCacheTarget();

core.debug(`Hashing target profile = ${cacheTarget}`);
hasher.update(cacheTarget);

const workflow = process.env.GITHUB_WORKFLOW;

if (workflow) {
core.debug(`Hashing GITHUB_WORKFLOW = ${workflow}`);
hasher.update(workflow);
}

const job = process.env.GITHUB_JOB;

if (job) {
core.debug(`Hashing GITHUB_JOB = ${job}`);
hasher.update(job);
}

return `${getCachePrefixes()[0]}-${hasher.digest('hex')}`;
}

export async function cleanCargoRegistry() {
core.info('Cleaning ~/.cargo before saving');

Expand Down Expand Up @@ -208,7 +153,7 @@ export async function cleanTargetProfile() {
}

export async function saveCache() {
if (!CACHE_ENABLED) {
if (!isCacheEnabled()) {
return;
}

Expand All @@ -229,7 +174,7 @@ export async function saveCache() {
}

export async function restoreCache() {
if (!CACHE_ENABLED) {
if (!isCacheEnabled()) {
return;
}

Expand Down