Working with GMS virtual machines (vagrant virtualbox)

Introduction

For instructions on how to install the GMS within a vagrant/virtualbox virtual machine, refer to the installation documentation.

A virtual machine (often called the "guest" system) exists within a conceptual sandbox on your physical machine (often called the "host" system). It shares resources of the host such as CPUs, memory, disk space, etc. with the host operating system. The hardware within the guest system is therefore fundamentally limited to that available on the host. The resources available to a virtual machine created using vagrant are configured by a configuration file called the 'vagrantfile'.

There are many platforms for creating virtual machines. This documentation describes the use of VirtualBox virtual machines. These virtual machines are automatically created using another platform called Vagrant. Vagrant allows us to fire up a virtual machine automatically at the command line without having to use a graphical user interface, wizards, etc. It is much faster to configure a virtual machine this way than it would be to manually walk through the process of install Ubuntu on a blank virtual machine.

We have no way to predict the resources available on your system. You should always consider adjusting the resources of your virtual machine based on what you have available on your host system.

The following are some notes on how to configure specific resources and work with GMS virtual machines created by Vagrant:

Modifying the GMS vagrant file and making changes take effect

Once you clone the GMS installation repo:

git clone https://github.com/genome/gms.git
cd gms

You can modify the vagrantfile Vagrantfile.standalone.virtualbox to configure host resources used by the virtual machine. In particular refer to the --memory and --cpus options in this file. If you change anything in this file and your virtual machine is already running you must restart it as follows at a terminal session (while within the directory containing your vagrantfile):

vagrant reload

or

vagrant halt
vagrant up

Note that using vagrant suspend and vagrant resume will pause your VM but new system configurations in your Vagrantfile will not take effect.

Adjusting memory resources available to the virtual machine

Some pipelines in the GMS can consume large amounts of memory. Mostly this occurs when many tasks are launched in parallel. You should determine the amount of memory available on your host system and set the memory available within your virtual machine appropriately. We recommend using up to 50-80% of the memory available on your host. Running the GMS in a virtual machine with less than 20Gb may affect performance. Adjusting memory available to the virtual machine is easy. For example, to increase the amount of memory from 10Gb to 24Gb, open the vagrant file and change this line:

v.customize ["modifyvm", :id, "--memory", 10000]

to this:

v.customize ["modifyvm", :id, "--memory", 24000]

Now reload the virtual machine as described above (vagrant reload) and then log in again (vagrant ssh) and check the memory available withing the virtual machine guest using top.

Adjust CPU resources available to the virtual machine

You should determine the amount of CPUs available on your host system and set the CPUs available within your virtual machine appropriately. We recommend using up to 50-75% of the CPUs available on your host. Running the GMS in a virtual machine with less than 4 CPUs may affect performance. Adjusting CPUs available to the virtual machine is easy. For example, to increase the amount of CPUs from 4 to 6, open the vagrant file and change this line:

v.customize ["modifyvm", :id, "--cpus", 4]

to this:

v.customize ["modifyvm", :id, "--cpus", 6]

Now reload the virtual machine as described above (vagrant reload) and then log in again (vagrant ssh) and check the memory available withing the virtual machine guest using top. Within the top interface type 1 to view statistics for each CPU.

Running multiple virtual machines concurrently on the same host system

If you want to run multiple GMS virtual machines concurrently you can clone the gms repo into multiple locations (e.g. 'gms1' and 'gms2') and install the GMS as normal in each of these. The one exception is that each of these must have a unique host port in their vagrant file Vagrantfile.standalone.virtualbox. For example,

gms1:

config.vm.network :forwarded_port, guest: 80, host: 9003

gms2:

config.vm.network :forwarded_port, guest: 80, host: 9004

Configuring disk storage of your virtual machine

To control the location of VDI images you will have to modify both Vagrantfile.standalone.virtualbox and setup/bin/vm-drive-setup-host.sh. For example, in Vagrantfile.standalone.virtualbox you might change vagrant_root + tmp-disk.vdi to /Volumes/Data1/tmp-disk.vdi. Similarly, in setup/bin/vm-drive-setup-host.sh you would replace all instances of tmp-disk.vdi with /Volumes/Data1/tmp-disk.vdi. And so on for other VDI locations.

Removing a GMS virtual machine installation

If you are testing the GMS installation you can start from scratch by simply exiting the VM, removing the VM by vagrant destroy in the directory with your vagrantfile and then simply deleting the entire directory where you cloned the GMS repo.