Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

All about debian #4

Merged
merged 2 commits into from
Mar 19, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
502 changes: 0 additions & 502 deletions Use Cases/Scenario 1 - NDVI Mapping/config.sample.yaml

This file was deleted.

2 changes: 1 addition & 1 deletion Use Cases/Scenario 1 - NDVI Mapping/integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
If Python is not available, install it with the package manager and do point 3 again.

```console
$ sudo yum install python3 python3-pip
$ sudo apt-get install -y python3 python3-pip
```

4. Check that GDAL is available.
Expand Down
1 change: 0 additions & 1 deletion Use Cases/Scenario 2 - Rapid Mapping/conda-install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ URL="https://github.com/conda-forge/miniforge/releases/download/${MINIFORGE_VERS
INSTALLER_PATH=/tmp/miniforge-installer.sh
# Make sure user's $HOME is not tampered with since this is run as root
unset HOME
yum install -y wget
wget --quiet $URL -O ${INSTALLER_PATH}
chmod +x ${INSTALLER_PATH}
# Check sha256 checksum
Expand Down
6 changes: 3 additions & 3 deletions Use Cases/Scenario 2 - Rapid Mapping/integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

1. Provision a virtual machine on the target site. Preferably with the following specification
- 2 CPUs, 8GB RAM, 30GB disk
- CentOS 7
- Debian 11+
- Jupyter Lab (with Python 3 support), it can also be installed if not present.
- With data offer access if required

Expand Down Expand Up @@ -227,8 +227,8 @@

```console
$ ls -l ~
-rw-rw-r--. 1 centos centos 16212154 Jun 24 17:37 burned_area_20210305_20210424.rgb.tif
-rw-rw-r--. 1 centos centos 130225905 Jun 24 17:38 burned_area_20210305_20210424.tif
-rw-rw-r--. 1 debian debian 16212154 Jun 24 17:37 burned_area_20210305_20210424.rgb.tif
-rw-rw-r--. 1 debian debian 130225905 Jun 24 17:38 burned_area_20210305_20210424.tif
```

7. Download them to your computer and open it with any tool that can visualise TIFF files. Verify that the band/layer `NBR` shows an RGB image of the burn severity of the area of interest. [100%]
Expand Down
2 changes: 1 addition & 1 deletion Use Cases/Scenario 3 - Mosaicking/conda-install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ URL="https://github.com/conda-forge/miniforge/releases/download/${MINIFORGE_VERS
INSTALLER_PATH=/tmp/miniforge-installer.sh
# Make sure user's $HOME is not tampered with since this is run as root
unset HOME
yum install -y wget
sudo apt-get install -y wget
wget --quiet $URL -O ${INSTALLER_PATH}
chmod +x ${INSTALLER_PATH}
# Check sha256 checksum
Expand Down
2 changes: 1 addition & 1 deletion Use Cases/Scenario 3 - Mosaicking/integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
1. Provision a virtual machine on the target site. Preferably with the following specification

- 2 CPUs, 8 GB RAM, 100 GB disk
- CentOS 7
- Debian 11+
- Python 3
- With data offer access if available

Expand Down
2 changes: 1 addition & 1 deletion Use Cases/Scenario 4 - Trends Mapping/conda-install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ URL="https://github.com/conda-forge/miniforge/releases/download/${MINIFORGE_VERS
INSTALLER_PATH=/tmp/miniforge-installer.sh
# Make sure user's $HOME is not tampered with since this is run as root
unset HOME
yum install -y wget
sudo apt-get install -y wget
wget --quiet $URL -O ${INSTALLER_PATH}
chmod +x ${INSTALLER_PATH}
# Check sha256 checksum
Expand Down
14 changes: 7 additions & 7 deletions Use Cases/Scenario 4 - Trends Mapping/integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
1. Provision a virtual machine on the target site. Preferably with the following specification

- 2 CPUs, 8 GB RAM, 100 GB disk
- CentOS 7
- debian 7
- Python 3
- With data offer access if available

Expand Down Expand Up @@ -191,12 +191,12 @@

```console
$ ls -l output_data/
-rw-rw-r--. 1 centos centos 1702 May 31 16:40 graph_S3A_SL_2_LST____20210425T110535_20210425T110835_20210426T211759_0179_071_094_2160_LN2_O_NT_004.xml
-rw-rw-r--. 1 centos centos 1702 May 31 16:37 graph_S3B_SL_2_LST____20210425T102904_20210425T103204_20210426T210428_0179_051_336_2340_LN2_O_NT_004.xml
-rw-rw-r--. 1 centos centos 294126 May 31 16:42 lst_S3A_SL_2_LST____20210425T110535_20210425T110835_20210426T211759_0179_071_094_2160_LN2_O_NT_004.tif
-rw-rw-r--. 1 centos centos 925298 May 31 16:40 lst_S3B_SL_2_LST____20210425T102904_20210425T103204_20210426T210428_0179_051_336_2340_LN2_O_NT_004.tif
-rw-rw-r--. 1 centos centos 12339468 May 31 16:42 rgba_S3A_SL_2_LST____20210425T110535_20210425T110835_20210426T211759_0179_071_094_2160_LN2_O_NT_004.tif
-rw-rw-r--. 1 centos centos 10451496 May 31 16:40 rgba_S3B_SL_2_LST____20210425T102904_20210425T103204_20210426T210428_0179_051_336_2340_LN2_O_NT_004.tif
-rw-rw-r--. 1 debian debian 1702 May 31 16:40 graph_S3A_SL_2_LST____20210425T110535_20210425T110835_20210426T211759_0179_071_094_2160_LN2_O_NT_004.xml
-rw-rw-r--. 1 debian debian 1702 May 31 16:37 graph_S3B_SL_2_LST____20210425T102904_20210425T103204_20210426T210428_0179_051_336_2340_LN2_O_NT_004.xml
-rw-rw-r--. 1 debian debian 294126 May 31 16:42 lst_S3A_SL_2_LST____20210425T110535_20210425T110835_20210426T211759_0179_071_094_2160_LN2_O_NT_004.tif
-rw-rw-r--. 1 debian debian 925298 May 31 16:40 lst_S3B_SL_2_LST____20210425T102904_20210425T103204_20210426T210428_0179_051_336_2340_LN2_O_NT_004.tif
-rw-rw-r--. 1 debian debian 12339468 May 31 16:42 rgba_S3A_SL_2_LST____20210425T110535_20210425T110835_20210426T211759_0179_071_094_2160_LN2_O_NT_004.tif
-rw-rw-r--. 1 debian debian 10451496 May 31 16:40 rgba_S3B_SL_2_LST____20210425T102904_20210425T103204_20210426T210428_0179_051_336_2340_LN2_O_NT_004.tif
```

7. Download the files `rgba_*.tif` onto your computer and open it with a tool such as QGIS. Verify that it shows the desired information (land surface temperature) of the area of interest. [100%]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ URL="https://github.com/conda-forge/miniforge/releases/download/${MINIFORGE_VERS
INSTALLER_PATH=/tmp/miniforge-installer.sh
# Make sure user's $HOME is not tampered with since this is run as root
unset HOME
yum install -y wget
sudo apt-get install -y wget
wget --quiet $URL -O ${INSTALLER_PATH}
chmod +x ${INSTALLER_PATH}
# Check sha256 checksum
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,13 @@

1. Provision a virtual machine on the target site. Preferably with the following specification:
- 4 CPU, 32GB RAM, 100GB disk
- CentOS 7
- Debian 11+ or similar (Ubuntu)
- With data offer access if required

2. Open a terminal on the provisioned machine and install some prequisites, in case they are not yet present on the machine.

```
sudo yum install -y vim tree wget unzip libgfortran-4.8.5
sudo apt-get install -y bc wget unzip libgfortran5-amd64-cross
```

3. Install, if necessary, **conda** on the virtual machine and create the conda environment. Conda is needed as the vehicle to install the SNAP toolbox.
Expand Down
16 changes: 2 additions & 14 deletions src/cdab-client/cdab-client
Original file line number Diff line number Diff line change
Expand Up @@ -27,20 +27,8 @@
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/cdab-client

if [[ -n "$@" && $@ == *"TS1"* ]]; then
newargs="$@"
newargs=${newargs/-tn/-ts}
newargs=${newargs/--target_name/-ts}
newargs=${newargs/--target_credentials/-tc}
newargs=${newargs/-tu/-te}
newargs=${newargs/-tsn/-n}
newargs=${newargs/--testsite_name/-n}
newargs=${newargs/--load_factor/-lf}

newargs=`echo ${newargs} | sed -e 's/\( -\w*\)\ /\1=/g'`

echo "Command: /usr/bin/cdab-remote-client $newargs" >&2

exec /usr/bin/cdab-remote-client $newargs
echo "Use cdab-remote-client for this test" >&2
exit 1
fi

exec /usr/lib/cdab-client/cdab-testtools "$@"
20 changes: 10 additions & 10 deletions src/cdab-client/config.sample.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -685,7 +685,7 @@ service_providers:
two_factor_authentication_key: "KEY"
vm_name: "cdab-test"
key_name: "cdab-cdse-vmkey"
image_name: "CentOS 7"
image_name: "Debian 12"
flavor_name: "hm.medium"
cost_monthly: 70.04
cost_hourly: 0.114
Expand Down Expand Up @@ -1352,7 +1352,7 @@ service_providers:
identity_api_version: 3
vm_name: "cdab-test"
key_name: "cdab-cdse-vmkey"
image_name: "Standard_CentOS_7_latest"
image_name: "Standard_Debian_12_latest"
# From T-Systems Elastic Cloud Server creation wizard
# https://open-telekom-cloud.com/en/prices/price-calculator
flavor_name: "s3.large.8"
Expand Down Expand Up @@ -1728,7 +1728,7 @@ service_providers:
key_name: "cdab-key-onda"
# Name of image to be used for new virtual machine
# (choose from Compute > Images on the OpenStack dashboard).
image_name: "Centos 7"
image_name: "Debian 12"
# Name of flavour for new virtual machine
# (check under Compute > Instances > Launch Instance > Flavours/Flavors on the OpenStack dashboard).
# From https://www.onda-dias.eu/cms/services/catalogues/virtual-servers
Expand Down Expand Up @@ -1767,7 +1767,7 @@ service_providers:
# (must correspond to public key in key_name).
private_key_file: "/home/jenkins/cdab-key-onda.pem"
# User on virtual machine for SSH connections.
remote_user: "centos"
remote_user: "debian"
#Create an external volume for docker image and test execution; this is useful for flavours that have very limited main disk.
# The size of the additional disk is 20 GB.
#use_volume: True
Expand Down Expand Up @@ -2169,7 +2169,7 @@ service_providers:
identity_api_version: 3
vm_name: "cdab-test"
key_name: "cdab-key-creodias"
image_name: "CentOS 7"
image_name: "Debian 12"
# From https://creodias.eu/price-list
# eo2.large = 2 vCPU, 8GB RAM, 32GB HD, 51.57 EUR per month, 0.084 EUR per hour
# eo2.xlarge = 4 vCPU, 16GB RAM, 64GB HD, 103.15 EUR per month, 0.169 EUR per hour
Expand Down Expand Up @@ -2366,7 +2366,7 @@ service_providers:
compute:
connector: google
region_name: "europe-west1-b"
image_name: "projects/centos-cloud/global/images/centos-7-v20220303"
image_name: "product/google/debian12"
#flavor_name: "projects/modular-buffer-235216/zones/europe-west1-b/machineTypes/n1-highmem-2"
# n1-highmem-2 = 2 vCPU, 13GB RAM
# n2-highmem-2 = 2 vCPU, 16GB RAM
Expand Down Expand Up @@ -2492,7 +2492,7 @@ service_providers:
flavor_name: "r5a.large"
security_group: "sg-0cc75812b649925ce"
private_key_file: "/home/jenkins/cdab-key-aws.pem"
remote_user: "centos"
remote_user: "debian"
# https://calculator.aws -> EC2
cost_monthly: 94.27
cost_hourly: 0.1291
Expand Down Expand Up @@ -3208,7 +3208,7 @@ service_providers:
vm_name: "cdab-test"
key_name: "cdab-key-wekeo"
private_key_file: "/home/jenkins/cdab-key-wekeo.pem"
image_name: "CentOS 7"
image_name: "Debian 12"
# hm.medium = 2 vCPU, 16GB RAM, 64GB HD
flavor_name: "hm.medium"
cost_monthly: 70.04
Expand Down Expand Up @@ -3285,8 +3285,8 @@ service_providers:
vm_name: "cdab-test"
image:
publisher: "OpenLogic"
offer: "CentOS"
sku: "7_8-gen2"
offer: "Debian"
sku: "12"
version: "latest"
# Memory optimized E2s v3 = 2 vCPU, 16GB RAM, 32GB HD
flavor_name: "Standard_E2s_v3"
Expand Down
22 changes: 1 addition & 21 deletions src/cdab-remote-client/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,26 +16,6 @@ The **cdab-remote-client** is a command line utility that performs a local CDAB

Note: The Openstack terminology is inconsistent and might be confusing; **servers** (as in the command-line client) and **instances** (as on the dashboard) refer to the same thing, **virtual machines**, as they are called below.

### Prerequisites

* CentOS 7
* Python 3
* OpenStack client (a Python module that provides a command-line tool for OpenStack access)
* The Amazon AWS SDK for Python
* The Google API Client Library for Python
* The Microsoft Azure libraries for Python
* Root or proper _sudo_ permissions

### Installation

* Add the terradue-stable-el7 repository configuration on YUM,
* Install the tool with:

```
$ sudo yum install cdab-remote-client -y
```


## Usage

The main parameters are shown on the help page.
Expand Down Expand Up @@ -166,7 +146,7 @@ Most of the values for the various keys can be obtained from the OpenStack dashb
* **project_id**: Name of the project on the Google Cloud Platform.
* **region_name**: Zone, e.g. *europe-west2-c*.
* **key_name**: This setting is not applicable, as Google creates all accounts for which there are keys defined and authorises all keys. Key pairs are defined under https://console.cloud.google.com/compute/metadata/sshKeys.
* **image_name**: Fully qualified name of boot disk image to be used for new virtual machine, e.g. *projects/centos-cloud/global/images/centos-7-v20200309*.
* **image_name**: Fully qualified name of boot disk image to be used for new virtual machine, e.g. *projects/debian-cloud/global/images/debian-12*.
* **flavor_name**: The machine type name name, e.g. *e2-standard-2*. Can also be specified, as in previous version, as fully qualified name of machine type for new virtual machine, e.g *projects/<project-name>/zones/europe-west2-c/machineTypes/e2-standard-2*.

##### Specific settings for Amazon AWS EC2
Expand Down
2 changes: 1 addition & 1 deletion src/cdab-remote-client/bin/cdab-remote-client
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@
export PYTHONIOENCODING='utf8'
command="python3 /usr/lib/cdab-remote-client/cdab-remote-client.py3";
for a in "$@"; do command="${command} \"${a/"/\\\\"/}\""; done

python3 /usr/lib/cdab-remote-client/cdab-remote-client.py3 "$@"
98 changes: 0 additions & 98 deletions src/cdab-remote-client/etc/config.yaml.sample

This file was deleted.

Loading
Loading