Skip to content

Commit

Permalink
fix myst reference, add missing config file
Browse files Browse the repository at this point in the history
  • Loading branch information
skirui-source committed Nov 22, 2023
1 parent 5ad5fc3 commit 5221666
Show file tree
Hide file tree
Showing 2 changed files with 58 additions and 4 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
#cloud-config


# Bootstrap
packages:
- apt-transport-https
- ca-certificates
- curl
- gnupg-agent
- software-properties-common
- ubuntu-drivers-common

# Enable ipv4 forwarding, required on CIS hardened machines
write_files:
- path: /etc/sysctl.d/enabled_ipv4_forwarding.conf
content: |
net.ipv4.conf.all.forwarding=1

# create the docker group
groups:
- docker

# Add default auto created user to docker group
system_info:
default_user:
groups: [docker]


runcmd:

# Install Docker
- curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
- add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
- apt-get update -y
- apt-get install -y docker-ce docker-ce-cli containerd.io
- systemctl start docker
- systemctl enable docker



# Install NVIDIA driver
- DEBIAN_FRONTEND=noninteractive ubuntu-drivers install

# Install NVIDIA docker
- curl -fsSL https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
- curl -s -L https://nvidia.github.io/nvidia-docker/$(. /etc/os-release;echo $ID$VERSION_ID)/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
- apt-get update -y
- apt-get install -y nvidia-docker2
- systemctl restart docker

# Attempt to run a RAPIDS container to download the container layers and decompress them
- 'docker run --net=host --gpus=all --shm-size=256m rapidsai/rapidsai:cuda11.2-runtime-ubuntu18.04-py3.8 dask-scheduler --version'
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@
"\n",
"**NOTE:** In this notebook, we will explore two possible ways to use `dask-cloudprovider` to run our workloads on Azure VM clusters:\n",
"\n",
"1. [Option 1](#-Option-1:-Use-an-Azure-marketplace-VM-image.-): Using an [Azure Marketplace image](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/nvidia.ngc_azure_17_11?tab=overview) made available for free from NVIDIA. The RAPIDS container will be subsequently downloaded once the VMs start up.\n",
"2. [Option 2](#-Option-2:-Set-up-an-Azure-Customized-VM.-): Using [`packer`](https://docs.microsoft.com/en-us/azure/virtual-machines/linux/build-image-with-packer) to create a custom VM image to be used in the cluster. This image will include the RAPIDS container, and having the container already inside the image should speed up the process of provisioning the cluster.\n",
"1. [Option 1](Use-an-Azure-marketplace-VM-image): Using an [Azure Marketplace image](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/nvidia.ngc_azure_17_11?tab=overview) made available for free from NVIDIA. The RAPIDS container will be subsequently downloaded once the VMs start up.\n",
"2. [Option 2](Set-up-an-Azure-Customized-VM): Using [`packer`](https://docs.microsoft.com/en-us/azure/virtual-machines/linux/build-image-with-packer) to create a custom VM image to be used in the cluster. This image will include the RAPIDS container, and having the container already inside the image should speed up the process of provisioning the cluster.\n",
"\n",
"#### You can either use Option 1 or use Option 2.\n",
"---"
Expand Down Expand Up @@ -324,6 +324,7 @@
"tags": []
},
"source": [
"(Set-up-an-Azure-Customized-VM)=\n",
"### <span style=\"color:#8735fb; font-size:20pt\"> Option 2: Set up an Azure Customized VM </span>\n",
"\n",
"#### If you already have a customized VM and you know its resource id, jump to [Step f. of Option 2](#f.-Set-up-customized-VM-information-and-clear-default-dask-config.) "
Expand Down Expand Up @@ -596,6 +597,7 @@
"tags": []
},
"source": [
"(Use-an-Azure-marketplace-VM-image)=\n",
"### <span style=\"color:#8735fb; font-size:20pt\"> Step 2.1: Start the VM Cluster in Azure </span>\n",
"\n",
"Here, if you have used Option 1, i.e., the NVIDIA VM image, pass an empty string for `vm_image` information. \n",
Expand Down Expand Up @@ -2509,9 +2511,9 @@
],
"metadata": {
"kernelspec": {
"display_name": "rapids-23.08",
"display_name": "rapids-23.10",
"language": "python",
"name": "rapids-23.08"
"name": "rapids-23.10"
},
"language_info": {
"codemirror_mode": {
Expand Down

0 comments on commit 5221666

Please sign in to comment.