Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Workbench session init container image #862

Merged
merged 10 commits into from
Oct 30, 2024
Merged
40 changes: 40 additions & 0 deletions .github/workflows/build-bake-preview.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -421,3 +421,43 @@ jobs:
dockerhub-token: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
snyk-org: ${{ secrets.SNYK_ORG }}
snyk-token: '${{ secrets.SNYK_TOKEN }}'

workbench-session-init-daily:
needs: [versions]
name: Workbench Session Init - Daily
runs-on: ubuntu-latest-4x

env:
target: "workbench-session-init-daily"
WORKBENCH_DAILY_VERSION: ${{ needs.versions.outputs.WORKBENCH_DAILY_VERSION }}
WORKBENCH_PREVIEW_VERSION: ${{ needs.versions.outputs.WORKBENCH_PREVIEW_VERSION }}
PACKAGE_MANAGER_DAILY_VERSION: ${{ needs.versions.outputs.PACKAGE_MANAGER_DAILY_VERSION }}
PACKAGE_MANAGER_PREVIEW_VERSION: ${{ needs.versions.outputs.PACKAGE_MANAGER_PREVIEW_VERSION }}
CONNECT_DAILY_VERSION: ${{ needs.versions.outputs.CONNECT_DAILY_VERSION }}
Comment on lines +434 to +436
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to specify the version for products we aren't building in the workflow?

I don't yet fully understand if all bake targets require all these env vars.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ianpittwood This is really a question for you.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. It's a bake limitation. It doesn't handle null values on build variables even if they're not part of the specified targets.

BRANCH: ${{ github.head_ref || github.ref_name }}

concurrency:
group: bake-workbench-session-init-daily-${{ github.ref }}
cancel-in-progress: true

steps:
- name: Checkout
uses: actions/checkout@v4

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
id: setup-buildx
with:
buildkitd-config: ./share/buildkitd.toml

- name: Build, Test, and Push
uses: ./.github/actions/bake-test-push
with:
target: ${{ env.target }}
bakefile: docker-bake.preview.hcl
push-image: ${{ github.ref == 'refs/heads/main' || github.ref == 'refs/heads/dev' || github.ref == 'refs/heads/dev-rspm' }}
ghcr-token: ${{ secrets.GITHUB_TOKEN }}
dockerhub-username: ${{ secrets.DOCKER_HUB_USERNAME }}
dockerhub-token: ${{ secrets.DOCKER_HUB_ACCESS_TOKEN }}
snyk-org: ${{ secrets.SNYK_ORG }}
snyk-token: '${{ secrets.SNYK_TOKEN }}'
1 change: 1 addition & 0 deletions .github/workflows/build-manual.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ on:
- workbench
- workbench-for-google-cloud-workstations
- workbench-for-microsoft-azure-ml
- workbench-session-init
type:
description: "The type of image being built."
required: false
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/lint.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ jobs:
- {product: 'connect-content-init', os: 'ubuntu2204'}
- {product: 'package-manager', os: 'ubuntu2204'}
- {product: 'r-session-complete', os: 'ubuntu2204'}
- {product: 'workbench-session-init', os: 'ubuntu2204'}
- {product: 'workbench-for-microsoft-azure-ml', os: 'ubuntu2204'}
- {product: 'content/base', os: 'ubuntu1804'}
- {product: 'content/base', os: 'ubuntu2204'}
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/update-readme.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ jobs:
- {prefix: '', repository: 'content-pro', readme_path: './content/pro/README.md'}
- {prefix: 'rstudio-', repository: 'package-manager', readme_path: './package-manager/README.md'}
- {prefix: '', repository: 'r-session-complete', readme_path: './r-session-complete/README.md'}
- {prefix: '', repository: 'workbench-session-init', readme_path: './workbench-session-init/README.md'}

steps:
- name: Check Out Repo
Expand Down
7 changes: 4 additions & 3 deletions Justfile
Original file line number Diff line number Diff line change
Expand Up @@ -295,16 +295,17 @@ update-rsw-versions:
r-session-complete/.env \
workbench-for-microsoft-azure-ml/.env \
r-session-complete/Dockerfile.ubuntu2204 \
r-session-complete/Dockerfile.centos7 \
workbench/Dockerfile.ubuntu2204 \
workbench-for-microsoft-azure-ml/Dockerfile.ubuntu2204
workbench-for-microsoft-azure-ml/Dockerfile.ubuntu2204 \
workbench-session-init/Dockerfile.ubuntu2204
sed {{ sed_vars }} "s/RSW_VERSION:.*/RSW_VERSION: {{ RSW_VERSION }}/g" docker-compose.yml
sed {{ sed_vars }} "s/rstudio\/rstudio-workbench:.*/rstudio\/rstudio-workbench:$(just _get-clean-version {{ RSW_VERSION }})/g" docker-compose.yml
sed {{ sed_vars }} "s/^RSW_VERSION := .*/RSW_VERSION := \"{{ RSW_VERSION }}\"/g" \
Justfile
sed {{ sed_vars }} "s/[0-9]\{4\}\.[0-9]\{1,2\}\.[0-9]\{1,2\}/`just _get-clean-version {{ RSW_VERSION }}`/g" \
workbench/README.md \
r-session-complete/README.md
r-session-complete/README.md \
workbench-session-init/README.md
awk -v new_version="{{ RSW_VERSION }}" '
/variable WORKBENCH_VERSION/ { print; getline; print " default = \"" new_version "\""; next }
{ print }
Expand Down
4 changes: 2 additions & 2 deletions ci.Justfile
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ get-product-args $PRODUCT $OS $VERSION $USE_S3="false" $BRANCH=`git branch --sho
RSW_DOWNLOAD_URL=$(just -f ci.Justfile _get-rsw-download-url preview $OS)
fi

if [[ $PRODUCT == "workbench" || $PRODUCT == "r-session-complete" || $PRODUCT == "workbench-for-microsoft-azure-ml" ]]; then
if [[ $PRODUCT == "workbench" || $PRODUCT == "workbench-session-init" || $PRODUCT == "r-session-complete" || $PRODUCT == "workbench-for-microsoft-azure-ml" ]]; then
SHORT_NAME="RSW"
elif [[ $PRODUCT == "connect" || $PRODUCT == "connect-content-init" ]]; then
SHORT_NAME="RSC"
Expand Down Expand Up @@ -158,7 +158,7 @@ get-prerelease-args $TYPE $PRODUCT $OS $VERSION $BRANCH=`git branch --show`:

RSW_DOWNLOAD_URL=$(just -f ci.Justfile _get-rsw-download-url $TYPE $OS)

if [[ $PRODUCT == "workbench" || $PRODUCT == "r-session-complete" || $PRODUCT == "workbench-for-microsoft-azure-ml" ]]; then
if [[ $PRODUCT == "workbench" || $PRODUCT == "workbench-session-init" || $PRODUCT == "r-session-complete" || $PRODUCT == "workbench-for-microsoft-azure-ml" ]]; then
SHORT_NAME="RSW"
elif [[ $PRODUCT == "connect" || $PRODUCT == "connect-content-init" ]]; then
SHORT_NAME="RSC"
Expand Down
26 changes: 26 additions & 0 deletions docker-bake.preview.hcl
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,14 @@ variable WORKBENCH_BUILD_MATRIX {
}
}

variable WORKBENCH_SESSION_INIT_BUILD_MATRIX {
default = {
builds = [
{os = "ubuntu2204"},
]
}
}

### Group definitions ###
group "default" {
targets = [
Expand All @@ -175,6 +183,7 @@ group "default" {
"r-session-complete-daily",
"workbench-preview",
"workbench-daily",
"workbench-session-init-daily",
]
}

Expand Down Expand Up @@ -450,3 +459,20 @@ target "workbench-preview" {
RSW_DOWNLOAD_URL = get_rsw_download_url(builds.os)
}
}

target "workbench-session-init-daily" {
inherits = ["base"]
target = "build"

name = "workbench-session-init-daily-${builds.os}-${replace(tag_safe_version(WORKBENCH_DAILY_VERSION), ".", "-")}"
tags = get_tags(builds.os, "workbench-session-init-preview", WORKBENCH_DAILY_VERSION, "daily")

dockerfile = "Dockerfile.${builds.os}"
context = "workbench-session-init"

matrix = WORKBENCH_SESSION_INIT_BUILD_MATRIX

args = {
RSW_VERSION = WORKBENCH_DAILY_VERSION
}
}
1 change: 1 addition & 0 deletions tools/dockerhub_clean.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
"rstudio-workbench",
"rstudio-workbench-for-microsoft-azure-ml",
"rstudio-workbench-preview",
"workbench-session-init-preview",
]


Expand Down
2 changes: 1 addition & 1 deletion tools/get-version.py
Original file line number Diff line number Diff line change
Expand Up @@ -97,7 +97,7 @@ def clean_product_selection(product: str) -> str:


def rstudio_workbench_daily():
version_json = download_json("https://dailies.rstudio.com/rstudio/cranberry-hibiscus/index.json")
version_json = download_json("https://dailies.rstudio.com/rstudio/kousa-dogwood/index.json")
return version_json['workbench']['platforms']['jammy-amd64']['version']


Expand Down
25 changes: 25 additions & 0 deletions workbench-session-init/Dockerfile.ubuntu2204
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
FROM ubuntu:22.04 AS build

# Install required tools:
# - ca-certificates installs necessary certificates to use cURL with HTTPS websites
# - curl is used to download the runtime tar.gz
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update && \
apt-get install -y --no-install-recommends ca-certificates curl && \
rm -rf /var/lib/apt/lists/*

ARG RSW_VERSION=2024.09.0+375.pro3

SHELL ["/bin/bash", "-o", "pipefail", "-c"]
RUN mkdir -p /pwb-staging && \
RSW_VERSION_URL=$(echo -n "${RSW_VERSION}" | sed 's/+/-/g') && \
echo "Downloading https://s3.amazonaws.com/rstudio-ide-build/session/multi/x86_64/rsp-session-multi-${RSW_VERSION_URL}-x86_64.tar.gz" && \
curl -fsSL -o /pwb-staging/rsp-session-multi.tar.gz "https://s3.amazonaws.com/rstudio-ide-build/session/multi/x86_64/rsp-session-multi-${RSW_VERSION_URL}-x86_64.tar.gz" && \
skyeturriff marked this conversation as resolved.
Show resolved Hide resolved
mkdir -p /opt/session-components && \
tar -C /opt/session-components -xf /pwb-staging/rsp-session-multi.tar.gz && \
chmod -R 755 /opt/session-components && \
rm -rf /pwb-staging

COPY --chmod=755 run.sh /usr/local/bin/run.sh

ENTRYPOINT ["/usr/local/bin/run.sh"]
4 changes: 4 additions & 0 deletions workbench-session-init/NEWS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# 2024.11.0

- Add NEWS.md
- Add daily builds
50 changes: 50 additions & 0 deletions workbench-session-init/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# Posit Workbench Session Init Container

This directory contains a Dockerfile and script that will create an init container to copy session runtime components from a release package into a target mount directory. This init container can be used to pull the session runtime components into another base sesssion image, which can then be used to run Workbench sessions.

## Quick reference

* Maintained by: [the Posit Docker team](https://github.com/rstudio/rstudio-docker-products)
* Where to get help: [our Github Issues page](https://github.com/rstudio/rstudio-docker-products/issues)
* Posit Workbench image: [Docker Hub](https://hub.docker.com/r/rstudio/rstudio-workbench)
* RStudio r-session-complete image: [Docker Hub](https://hub.docker.com/r/rstudio/r-session-complete)
* Workbench Session Init image (Daily/Preview): [Docker Hub](https://hub.docker.com/r/rstudio/rstudio-workbench-session-init-preview)

## Supported tags and respective Dockerfile links

* [`jammy-daily`, `ubuntu2204-daily`, `jammy-2024.11.0`, `ubuntu2204-2024.11.0`](https://github.com/rstudio/rstudio-docker-products/blob/main/workbench-session-init/Dockerfile.2204)

## Building

Currently daily builds are supported. To build the image, run:

```console
just preview-bake workbench-session-init-daily
```

## Testing

You can observe what gets copied by the container:

```console
mkdir init
docker run --rm -v $(pwd)/init:/mnt/init rstudio/workbench-session-init-preview:workbench-session-init-jammy-2024.11.0-daily-328.pro3
# The init directory has been populated with the Workbench session runtime components.
```

You can also test using GOSS:

```console
just preview-test workbench-session-init-daily
```

## Licensing

The license associated with the RStudio Docker Products repository is located [in LICENSE.md](https://github.com/rstudio/rstudio-docker-products/blob/main/LICENSE.md).

As is the case with all container images, the images themselves also contain other software which may be under other
licenses (i.e. bash, linux, system libraries, etc., along with any other direct or indirect dependencies of the primary
software being contained).

It is an image user's responsibility to ensure that use of this image (and any of its dependent layers) complies with
all relevant licenses for the software contained in the image.
17 changes: 17 additions & 0 deletions workbench-session-init/run.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
#!/usr/bin/env bash

set -e
skyeturriff marked this conversation as resolved.
Show resolved Hide resolved
set -x

S=/opt/session-components

# The target should exist and be an empty directory.
T=/mnt/init

if [ ! -d "${T}" ] ; then
echo "Cannot find the copy target ${T}"
exit 1
fi

echo "Copying files from /session-components to /mnt/init"
time cp -r $S/* $T
13 changes: 13 additions & 0 deletions workbench-session-init/test/goss.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
package:
curl:
installed: true

file:
/opt/session-components/:
exists: true
mode: "0755"
filetype: directory
Comment on lines +6 to +9
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there are specific session component files, we should probably check for them here.

If we want to check the indivdual contents of a tarfile that was added to the container, we could use a command block, running tar with the -t flag to ensure that the tarball has the expected contents.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ooh, I think the issue here is the tarball gets cleaned up in the last line of the Dockerfile RUN command, so it won't be around for goss to test against. But I think it's good practice to verify this, so I could check against the extracted directory structure. I'm uncertain how this would behave across versions though, if say we removed or added contents to the archive in a future release. I think in that case, we'd need to update the test, which would then only run successfully against the latest version?

/usr/local/bin/run.sh:
exists: true
filetype: file
mode: "0755"
20 changes: 20 additions & 0 deletions workbench-session-init/test/run_tests.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
#!/bin/bash

# install goss
GOSS_FILE=${GOSS_FILE:-/test/goss.yaml}
GOSS_VERSION=${GOSS_VERSION:-0.4.6}
GOSS_MAX_CONCURRENT=${GOSS_MAX_CONCURRENT:-50}

if [ -f /etc/debian_version ]; then
OS="ubuntu"
else
echo "OS not supported. Exiting"
exit 1
fi

# install goss to tmp location and make executable
curl -fsSL https://github.com/aelsabbahy/goss/releases/download/v$GOSS_VERSION/goss-linux-amd64 -o /tmp/goss \
&& chmod +x /tmp/goss \
&& GOSS=/tmp/goss

OS=$OS GOSS_FILE=$GOSS_FILE $GOSS v --format documentation --max-concurrent $GOSS_MAX_CONCURRENT