Skip to content

Commit

Permalink
Merge branch 'dev' into 413-cast-config-returns-to-appropriate-types
Browse files Browse the repository at this point in the history
  • Loading branch information
@rowan04
rowan04 authored Sep 29, 2023
2 parents 89756eb + 7bc3b3f commit 08b68c0
Show file tree
Hide file tree
Showing 298 changed files with 8,882 additions and 6,248 deletions.
3 changes: 3 additions & 0 deletions .codeclimate.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@ version: "2"
plugins:
phpcodesniffer:
enabled: true
# Need the beta channel to enforce PSR12,
# see https://docs.codeclimate.com/docs/phpcodesniffer#standard.
channel: "beta"
config:
standard: "phpcs.xml"
phpmd:
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/continuous-integration.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ jobs:

steps:
- name: "Checkout"
uses: "actions/checkout@v3"
uses: "actions/checkout@v4"
with:
fetch-depth: 2

Expand Down Expand Up @@ -106,7 +106,7 @@ jobs:

steps:
- name: "Checkout"
uses: "actions/checkout@v3"
uses: "actions/checkout@v4"
with:
fetch-depth: 2

Expand Down
89 changes: 48 additions & 41 deletions INSTALL.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ This file is best viewed using a browser-plugin for markdown `.md` files.

* [Database server](#database-server)
* Oracle 11g+ or MariaDB/MySQL
* (note: the free Oracle 11g XE Express Editions which comes with a free license is perfectly suitable)
* Oracle 18c XE Express Edition available under the Oracle Free Use Terms and Conditions is suitable for development and testing (https://www.oracle.com/downloads/licenses/oracle-free-license.html).
* MariaDB/MySQL
* package `mariadb-server`

Expand Down Expand Up @@ -117,13 +117,23 @@ For GocDB, three URL alias/directory-mappings are needed, one for the portal GUI
Note that, depending on Apache/httpd version, the "Require all granted" statements in gocdbssl.conf may cause an HTTP Error "500 - Invalid configuration..." and can be commented out.

### Database Server
GOCDB uses a DB abstraction layer (Doctrine) and with some configuration should be deployable on different RDBMS platforms that are supported for Doctrine. Instructions are provided here for Oracle (the free Oracle 11g is perfectly suitable) and MySQL/MariaDB.
GOCDB uses a DB abstraction layer (Doctrine) and with some configuration should be deployable on different RDBMS platforms that are supported for Doctrine. Instructions are provided here for Oracle and MySQL/MariaDB.

#### Oracle 11g
The free to use XE/11g Oracle DB can be used to host run GOCDB on Win/nix. To use Oracle on nix systems, the OCI8 extension/driver needs to be compiled and installed.
#### Oracle XE (18c)
The free to use XE/18c Oracle DB can be used to host the GOCDB database for development and testing. This document adapts the instructions used for the unsupported Oracle 11g database. However, using a single, 'common user' at the root of a multitenant container database is not necessarily the best, or most secure, configuration.

To download and install the database server, follow the instructions linked from here - https://www.oracle.com/database/technologies/appdev/xe.html.

Notes on Oracle 18c XE installation:
- "SEVERE: [FATAL] [DBT-06103] The port (5,500) is already in use." is caused by the short host name (alias) not being configured in /etc/hosts. (https://dba.stackexchange.com/questions/268437/fatal-dbt-06103-the-port-5-500-is-already-in-use)
- PATH Bash environment variable should be updated to include the Oracle executables BEFORE the oraenv script is run -
```
export PATH=/opt/oracle/product/18c/dbhomeXE/bin:$PATH
```

#### Compiling/Installing OCI8
Using Oracle on Linux requires the OCI8 extension/driver for php: http://php.net/oci8.

##### Compiling/Installing OCI8
The OCI8 extension/driver for php needs to be installed, see: http://php.net/oci8
This can be most easily installed with the free Oracle Instant Client libs which can be installed in a number of ways (http://php.net/manual/en/oci8.installation.php), but the most easy is via PECL as descibed below:

Install the basic, devel and sqlplus instantclient rpms from Oracle (http://www.oracle.com/technetwork/database/features/instant-client/index-097480.html) and install GCC, PHP dev and pear packages:
Expand Down Expand Up @@ -277,36 +287,33 @@ deployment of your GOCDB instance:
* [Deploy Sample Data](#deploy-sample-data) (optional)

### Create DB User/Account <a id="create-db"></a>
#### Oracle
#### Oracle <a id="oracle-user"></a>

If you intend to populate the database from a dump of an existing GOCDB5 instance you do NOT need to create the GOCDB5 user. Simply deploy the data as described at ["Deploy and existing DB"](#deploy-existing-dump) below remembering that you might want to ALTER the password for the GOCDB5 user after the import.
If you intend to populate the database from a dump of an existing GOCDB5 instance you do NOT need to create the GOCDB5 user. Simply deploy the data as described at ["Deploy and existing DB"](#deploy-existing-dump).

Create a dedicated GOCDB5 user using the following script (substitute GOCDB5 for your username and a secure password). Run this script as the Oracle admin/system user:
Create a dedicated GOCDB5 user using the following script (substitute C##GOCDB5 for your username and a secure password). The C## (or c##) prefix is required to indicate an Oracle 'common user'. Run this script as the Oracle admin/system user:

```
-- Manage GOCDB5 user if already exists (optional) --
drop user gocdb5 cascade;
-- CREATE USER SQL
CREATE USER GOCDB5 IDENTIFIED BY <PASSWORD>
DROP USER C##GOCDB5 CASCADE;
CREATE USER C##GOCDB5 IDENTIFIED BY <PASSWORD>
DEFAULT TABLESPACE "USERS"
QUOTA UNLIMITED ON "USERS"
TEMPORARY TABLESPACE "TEMP";
-- ROLES - GRANT "RESOURCE" TO GOCDB5
-- SYSTEM PRIVILEGES
GRANT CREATE TRIGGER TO GOCDB5 ;
GRANT CREATE SEQUENCE TO GOCDB5 ;
GRANT CREATE TABLE TO GOCDB5 ;
GRANT CREATE JOB TO GOCDB5 ;
GRANT CREATE PROCEDURE TO GOCDB5 ;
GRANT CREATE TYPE TO GOCDB5 ;
GRANT CREATE SESSION TO GOCDB5 ;
--
GRANT CREATE TRIGGER TO C##GOCDB5;
GRANT CREATE SEQUENCE TO C##GOCDB5;
GRANT CREATE TABLE TO C##GOCDB5;
GRANT CREATE JOB TO C##GOCDB5;
GRANT CREATE PROCEDURE TO C##GOCDB5;
GRANT CREATE TYPE TO C##GOCDB5;
GRANT CREATE SESSION TO C##GOCDB5;
```

If you are using sqlplus to connect to the database remotely you will need also -

```
GRANT CONNECT TO GOCDB5;
GRANT CONNECT TO C##GOCDB5;
```

By default, Oracle 11g will expire a password in 180 days. In previous versions
Expand Down Expand Up @@ -424,21 +431,18 @@ $ php deploy/DeployRequiredDataRunner.php requiredData

### OPTIONAL: Deploy Sample Data<a id="deploy-sample-data"></a>

You can choose to deploy some sample data to seed your DB with sample users,
sites and services. Two sample data sets are available. Choose one of -

1. Minimal - just enough to get going with no real-world associations.

```bash
$ cd lib/Doctrine
$ php deploy/DeploySampleDataRunner.php simpleSampleData
```
1. "Real World" - a small subset derived from real data.
You can choose to deploy some sample data to seed your database. It consists of
a project which contains 2 NGIs, and another NGI not in a project. Each NGI contains
multiple sites, and each site has 1-5 services. Each service belongs to at least one
service group, of which there are 7. Each service group has 5-8 services. The
database is populated with unique sample users, which have roles over relevant site,
NGI, project and service group entities. The sample data has no real-world
associations.

```bash
$ cd lib/Doctrine
$ php deploy/DeploySampleDataRunner.php sampleData
```
```bash
$ cd lib/Doctrine
$ php deploy/DeploySampleDataRunner.php sampleData
```

### ORACLE ONLY: Deploy an existing DB .dmp file to populate your DB<a id="deploy-existing-dump"></a>

Expand All @@ -457,13 +461,11 @@ This directory object defines the directory where the .dmp file is loaded from.
```

* Import your dmp file. Note, the example below assumes the 'gocdb5' user/schema does not exist in the db - the import actually creates this user with all its permissions/roles.
If you want to use a different schema/username, then specify this in the value of the remap_schema argument on the right of the colon.
You may need to change different arguments for your install such as modifying the remap_tablespace:
If you want to use a different schema/username, then specify this in the value of the remap_schema argument on the right of the colon. (The following example remaps to an Oracle 'common user' as described in [creating the Oracle user/account](#oracle-user) above, after importing an Oracle 11 dump to an Oracle 18 service). You may need to change different arguments for your install such as modifying the remap_tablespace:

```
$impdp system/******** schemas=gocdb5 directory=dmpdir dumpfile=goc5dump.dmp REMAP_SCHEMA=gocdb5:gocdb5 remap_tablespace=GOCDB5:users table_exists_action=replace logfile=gocdbv5deploy.log
$impdp system/******** schemas=gocdb5 directory=dmpdir dumpfile=goc5dump.dmp REMAP_SCHEMA=gocdb5:c##gocdb5 remap_tablespace=GOCDB5:users table_exists_action=replace logfile=gocdbv5deploy.log
```
Note: If you get the following error, there is a file permissionsissue of some kind.
Try creating a new directory for the dump-file, possibly within your Oracle directory.

Expand All @@ -474,6 +476,11 @@ You may need to change different arguments for your install such as modifying th
ORA-06512: at "SYS.UTL_FILE", line 536
ORA-29283: invalid file operation
```
* Change the database password if necessary -

```
SQL> alter user c##gocdb5 identified by <password>;
```

* To generate statistics after importing the dmp file (this improves performance):

Expand Down
13 changes: 8 additions & 5 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,16 @@
Grid Operations Configuration Management Database. A Repository, Portal and REST style API for managing Grid and Cloud topology objects including; projects, administrative domains, sites, services, service-endpoints, service-groups, downtimes, users, roles and business rules.

<span>
<img alt="STFC logo" src="htdocs/images/UKRI_STF_Council-Logo_Horiz-RGB_crop.png" height=57/>
<img alt="EU flag" src="htdocs/images/eu_flag_yellow_low_150.png" height=57 />
<img alt="EGI logo" src="htdocs/images/egi_logo.jpg" height=57 />
<img alt="EOSC-hub logo" src="htdocs/images/eosc-hub-v-web_150.png" height=57 />
<img alt="STFC logo" src="htdocs/images/logos/ukri_stfc.png" height=64/>
<img alt="EU flag" src="htdocs/images/flags/eu.png" height=57 />
<img alt="EGI logo" src="htdocs/images/logos/egi.png" height=57 />
<img alt="EOSC Future logo" src="htdocs/images/logos/eosc_future.png" height=57 />
<img alt="IRIS logo" src="htdocs/images/logos/iris_ac_uk.png" height=57 />
</span>

GOCDB is provided by [STFC](https://stfc.ukri.org/) for [EGI](https://www.egi.eu/federation/), co-funded by [EGI.eu](https://www.egi.eu/) and [EOSC-hub](https://www.eosc-hub.eu/). Licensed under the [Apache 2 License](http://www.apache.org/licenses/LICENSE-2.0).
GOCDB is provided by [STFC](https://stfc.ukri.org/), part of [UK Research and Innovation](https://www.ukri.org/), and is co-funded by [EGI](https://egi.eu) via [EGI-ACE](https://www.egi.eu/project/egi-ace/), [EOSC-Future](https://eoscfuture.eu/) and the [IRIS](https://www.iris.ac.uk/) community.

Licensed under the [Apache 2 License](https://www.apache.org/licenses/LICENSE-2.0).

## Documentation

Expand Down
28 changes: 16 additions & 12 deletions config/gocdb_schema.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -415,18 +415,22 @@
</entity>
<!-- ========================================================== -->
<entity>
<name>service_type</name>
<field>
<fname>NAME</fname>
<length>255</length>
<regex>/^([[:alpha:]]|[[:digit:]]|[\._-])*$/</regex>
</field>
<field>
<fname>DESCRIPTION</fname>
<length>255</length>
<!--<regex>/^[a-zA-Z0-9\-\._\(\)\[\],;+:\/'"\s]*$/</regex>-->
<regex>/^[^`'\";&lt;&gt;]+$/</regex>
</field>
<name>service_type</name>
<field>
<fname>NAME</fname>
<length>255</length>
<regex>/^([[:alpha:]]|[[:digit:]]|[\._-])*$/</regex>
</field>
<field>
<fname>DESCRIPTION</fname>
<length>255</length>
<!--<regex>/^[a-zA-Z0-9\-\._\(\)\[\],;+:\/'"\s]*$/</regex>-->
<regex>/^[^`'\";&lt;&gt;]+$/</regex>
</field>
<field>
<fname>ALLOWMONITORINGEXCEPTION</fname>
<ftype>boolean</ftype>
</field>
</entity>
<!-- ========================================================== -->
<entity>
Expand Down
2 changes: 2 additions & 0 deletions config/local_info.xml
View file
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
<?xml version="1.0" encoding="UTF-8"?>

<config>
<!--
Provide default values for all configuration elements.
See also local override information provided below this local_info element.
-->

<local_info>

<!-- If read-only is set as true, the portal will prevent changes to the
Expand Down
132 changes: 132 additions & 0 deletions config/local_info.xsd
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
<?xml version="1.0"?>

<!-- Copied from https://www.ibm.com/developerworks/library/x-validxphp/ -->
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema">

<xs:include schemaLocation="./web_portal/menu.xsd"/>

<xs:simpleType name="showType">
<xs:restriction base="xs:string">
<xs:enumeration value="hide"/>
<xs:enumeration value="show"/>
<xs:enumeration value="no"/>
<xs:enumeration value="yes"/>
<xs:enumeration value="false"/>
<xs:enumeration value="true"/>
</xs:restriction>
</xs:simpleType>

<xs:element name="config">
<xs:complexType>
<xs:sequence>
<xs:element name="local_info" minOccurs="1" maxOccurs="unbounded">
<xs:complexType>
<xs:all>

<xs:element name="page_banner" type="xs:string" minOccurs="0"/>
<xs:element name="read_only" type="xs:boolean" minOccurs="0"/>
<xs:element name="filter_downtimes_by_default_scope" type="xs:boolean" minOccurs="0"/>
<xs:element name="default_filter_by_scope" type="xs:boolean" minOccurs="0"/>
<xs:element name="menus" type="validMenus" minOccurs="0"/>

<!-- Note that we do not validate the email string values provided here -->
<xs:element name="email_from" type="xs:string" minOccurs="0"/>
<xs:element name="email_to" type="xs:string" minOccurs="0"/>

<xs:element name="aup" type="xs:anyURI" minOccurs="0"/>
<xs:element name="aup_title" type="xs:string" minOccurs="0"/>
<xs:element name="privacy_notice" type="xs:anyURI" minOccurs="0"/>
<xs:element name="privacy_notice_title" type="xs:string" minOccurs="0"/>

<xs:element name="restrict_personal_data" type="xs:boolean" minOccurs="0"/>

<xs:element name="API_all_auth_realms" type="xs:boolean" minOccurs="0"/>

<xs:element name="css" minOccurs="0">
<xs:complexType>
<xs:all>
<!-- Use string type to allow standard colour names -->
<xs:element name="backgroundColour1" type="xs:string"/>
<xs:element name="backgroundColour2" type="xs:string"/>
<xs:element name="backgroundColour3" type="xs:string"/>
<xs:element name="backgroundDirection" type="xs:string"/>
<xs:element name="headingTextColour" type="xs:string"/>

</xs:all>
</xs:complexType>
</xs:element>
<xs:element name="name_mapping" minOccurs="0">
<xs:complexType>
<xs:all>
<xs:element name="Service">
<xs:complexType>
<xs:all>
<xs:element name="ServiceEndpoint" type="xs:string" minOccurs="0"/>
<xs:element name="endpoint" type="xs:string" minOccurs="0"/>
</xs:all>
</xs:complexType>
</xs:element>
</xs:all>
</xs:complexType>
</xs:element>
<xs:element name="web_portal_url" type="xs:anyURI" minOccurs="0"/>
<xs:element name="pi_url" type="xs:anyURI" minOccurs="0"/>
<xs:element name="server_base_url" type="xs:anyURI" minOccurs="0"/>
<xs:element name="write_api_user_docs_url" type="xs:anyURI" minOccurs="0"/>
<xs:element name="optional_features" minOccurs="0">
<xs:complexType>
<xs:all>
<xs:element name="siteless_services" type="xs:boolean"/>
</xs:all>
</xs:complexType>
</xs:element>
<xs:element name="default_scope" minOccurs="0">
<xs:complexType>
<xs:all>
<xs:element name="name" type="xs:string"/>
</xs:all>
</xs:complexType>
</xs:element>
<xs:element name="show_map_on_start_page" type="xs:boolean" minOccurs="0"/>
<xs:element name="default_scope_match" minOccurs="0">
<xs:simpleType>
<xs:restriction base="xs:string">
<xs:enumeration value="any"/>
<xs:enumeration value="all"/>
</xs:restriction>
</xs:simpleType>
</xs:element>
<xs:element name="minimum_scopes" minOccurs="0">
<xs:complexType>
<xs:all>
<xs:element name="ngi" type="xs:integer" minOccurs="0"/>
<xs:element name="site" type="xs:integer" minOccurs="0"/>
<xs:element name="service" type="xs:integer" minOccurs="0"/>
<xs:element name="service_group" type="xs:integer" minOccurs="0"/>
</xs:all>
</xs:complexType>
</xs:element>
<xs:element name="reserved_scopes" minOccurs="0">
<xs:complexType>
<xs:sequence>
<xs:element name="scope" type="xs:string" minOccurs="1" maxOccurs='unbounded'/>
</xs:sequence>
</xs:complexType>
</xs:element>
<xs:element name="extensions" minOccurs="0">
<xs:complexType>
<xs:all>
<xs:element name="max" type="xs:integer"/>
</xs:all>
</xs:complexType>
</xs:element>
<xs:element name="send_email" type="xs:boolean" minOccurs="0"/>
</xs:all>
<xs:attribute name="url" type="xs:anyURI"/>
</xs:complexType>
</xs:element>
</xs:sequence>
</xs:complexType>
</xs:element>

</xs:schema>
Loading

0 comments on commit 08b68c0

Please sign in to comment.