`_
+
+
+Job Templates Overview
+......................
+
+"Job Composer" attempts to model a simple but common workflow. When creating a new batch job to run a simulation a user may:
+
+1. copy the directory of a job they already ran or an example job
+2. edit the files
+3. submit a new job
+
+"Job Composer" implements these steps by providing the user job template directories and the ability to make copies of them: (1) Copy a directory, (2) Edit the files, and (3) Submit a new job.
+
+1. Copy a directory of a job already ran or an example job
+
+ 1. User can create a new job from a "default" template. A custom default template can be defined at ``/etc/ood/config/apps/myjobs/templates/default`` or under the app deployment directory at ``/var/www/ood/apps/sys/myjobs/templates/default``. If no default template is specified, the default is ``/var/www/ood/apps/sys/myjobs/example_templates/torque``
+ 2. user can select a directory to copy from a list of "System" templates the admin copied to ``/etc/ood/config/apps/myjobs/templates`` or under the app deployment directory at ``/var/www/ood/apps/sys/myjobs/templates`` during installation
+ 3. user can select a directory to copy from a list of "User" templates that the user has copied to ``$HOME/ondemand/data/sys/myjobs/templates``
+ 4. user can select a job directory to copy that they already created through "Job Composer" from ``$HOME/ondemand/data/sys/myjobs/projects/default``
+
+2. Edit the files
+
+ 1. user can open the copied job directory in the File Explorer and edit files using the File Editor
+
+3. Submit a new job
+
+ 1. user can use the Job Options form specify which host to submit to, what file is the job script
+ 2. user can use the web interface to submit the job to the batch system
+ 3. after the job is completed, the user can open the directory in the file explorer to view results
+
+Job Template Details
+....................
+
+A template consists of a folder and a `manifest.yml` file.
+
+The folder contains files and scripts related to the job.
+
+The manifest contains additional metadata about a job, such as a name, the default host, the submit script file name, and any notes about the template.
+
+.. code:: yaml
+
+ name: A Template Name
+ host: ruby
+ script: ruby.sh
+ notes: Notes about the template, such as content and function.
+
+In the event that a job is created from a template that is missing from the `manifest.yml`, "Job Composer" will assign the following default values:
+
+- ``name`` The name of the template folder.
+- ``host`` The cluster id of the first cluster with a valid resource_mgr listed in the OOD cluster config
+- ``script`` The first ``.sh`` file appearing in the template folder.
+- ``notes`` The path to the location where a template manifest should be located.
+
+Job Composer Script Size Limit
+------------------------------
+
+Since 1.7 the Job composer shows users 'Suggested file(s)' and 'Other valid file(s)'. Other valid files are
+_any_ files less than ``OOD_MAX_SCRIPT_SIZE_KB`` which defaults to 65 (meaning 65kb).
+
+To reconfigure this, simply set the environment variable in the job composers' env file
+``/etc/ood/config/apps/myjobs/env`` like so:
+
+.. code:: sh
+
+ # show any file less than or equal to 15 kb
+ OOD_MAX_SCRIPT_SIZE_KB=15
+
+Hiding Job Arrays
+------------------------------
+
+When composing a new job, the job arrays field is shown on supported clusters. To Hide this field even on
+supported clusters, an option was added.
+
+To reconfigure this, simply set the environment variable in the job composers' env file
+``/etc/ood/config/apps/myjobs/env`` like so:
+
+.. code:: sh
+
+ # Don't show job arrays field even on supported clusters
+ OOD_HIDE_JOB_ARRAYS=True
+
+Custom Error Page for Missing Home Directory on Launch
+------------------------------------------------------
+
+Some sites have the home directory auto-create on first ssh login, for example
+via ``pam_mkhomedir.so``. This introduces a problem if users first access the system
+through OnDemand, which expects the existence of a user’s home directory.
+
+In OnDemand <= 1.3 if the user's home directory was missing a non-helpful single
+string error would display. Now a friendly error page displays. This error page
+can be customized by adding a custom one to ``/etc/ood/config/pun/html/missing_home_directory.html``.
+
+The default error page looks like this:
+
+.. figure:: /images/customization_homedirmissing_default.png
+ :align: center
+
+An example of a custom error page has been provided at ``/opt/ood/nginx_stage/html/missing_home_directory.html.example.pam_mkhomedir`` and can be copied to ``/etc/ood/config/pun/html/missing_home_directory.html``. This example directs the user to first click a link to open the shell app which will create the home directory. The shell app's default host must be configured to be a host that is appropriate for this purpose. The custom error page looks like this:
+
+.. figure:: /images/customization_homedirmissing_pammkdir.png
+ :align: center
+
+
+
+See `this Discourse discussion `_ for details.
+
+.. _dashboard_pinned_apps:
+
+Pinning Applications to the Dashboard
+-------------------------------------
+
+In version 2.0 you can now pin app Icons to the dashboard that link to the application form.
+
+When configured a widget like the one below will appear on the dashboard's landing page.
+
+.. figure:: /images/pinned_apps.png
+
+The configuration for what apps to pin allows for three variants.
+
+You can configure specific apps with a string of the type ``router/app_name``.
+For example ``sys/jupyter`` is the system installed app named jupyter.
+
+Secondly you can configure globs like ``sys/*`` to pin all system installed apps. Or
+Maybe ``sys/minimal_*`` to pin all system installed apps that begin with 'minimal'.
+
+Lastly you can choose to pin apps based off of fields in their ``manifest.yml`` file.
+You can match by type, category, subcategory and metadata fields. These matches are
+cumulative. Meaning an app has to match *all* of these to be pinned. In the examples below
+there is a configuration of type sys and category minimal. This configuration will only pin
+system installed apps that are in the minimal category. An app has to meet *both* these
+criteria to be pinned to the dashboard.
+
+
+Full examples are below:
+
+.. code:: yaml
+
+ # /etc/ood/config/ondemand.d/ondemand.yml
+ pinned_apps:
+ - sys/jupyter # pin a specific system installed app called 'jupyter'
+
+ - sys/bc_desktop/desk1 # pinned desktop app must contain exact desktop name - 'desk1'
+
+ - 'sys/*' # pin all system install apps. This also works for usr/* and dev/*
+
+ - category: 'minimal' # pin all the apps in the 'minimal' category
+
+ - type: sys # pin all system installed apps in the minimal category.
+ category: 'minimal'
+
+ # pin all system installed apps in the minimal category and the
+ # class instruction subcategory
+ - type: sys
+ category: 'minimal'
+ subcategory: 'class_instruction'
+
+ # pin all system installed apps in the minimal category, the
+ # class instruction subcategory and the metadata field 'field_of_science'
+ # with an exact match on biology
+ - type: sys
+ category: 'minimal'
+ subcategory: 'class_instruction'
+ field_of_science: 'biology'
+
+ # pin any app with an exact match on the metadata field_of_science of biology
+ - field_of_science: 'biology'
+
+ # pin any app with a glob match *bio* on the metadata field_of_science
+ - field_of_science: '*bio*'
+
+
+Administrators can also configure the pinned apps to be grouped by any field
+in the ``manifest.yml`` including metadata fields with the ``pinned_apps_group_by``
+configuration.
+
+This will create a row and a heading for each group like so (the image was generated
+from grouping by category):
+
+.. figure:: /images/grouped_pinned_apps.png
+
+One can also change the menu length in the 'App's menu item. If you've
+pinned more than 6 apps and you want to them to show up in this dropdown
+list, simply increase the length with the option below.
+
+.. code:: yaml
+
+ # /etc/ood/config/ondemand.d/ondemand.yml
+ pinned_apps_menu_length: 6 # the default number of items in the dropdown menu list
+ pinned_apps_group_by: category # defaults to nil, no grouping
+
+Pinned Apps customizations
+..........................
+
+To customize the text, icon, or color of the pinned app tile,
+use the ``tile`` configuration property in the application ``manifest`` or ``form``.
+The ``form`` values will take precedence over any set in the ``manifest``.
+All the values are optional and any set will override the default from the application.
+
+.. code:: yaml
+
+ tile:
+ title: "Custom Title"
+ icon: fa://desktop
+ border_color: "red"
+ sub_caption: |
+ Custom Text Line 1
+ Text Line 2
+ Text Line 3
+
+The CSS for the pinned app tiles has been optimized to support upto three lines of text for the ``sub_caption`` property.
+
+.. figure:: /images/custom_pinned_apps.png
+
+.. _dashboard_custom_layout:
+
+Custom layouts in the dashboard
+-------------------------------
+
+Administrators can now customize what widgets appear on the dashboard and how they're
+laid out on the page.
+
+In it's simplest form this feature allows for a rearrangement of existing widgets. As
+of 2.1 the existing widgets are:
+
+- ``pinned_apps`` - Pinned Apps described above
+- ``recently_used_apps`` - the four most recently used interactive applications.
+ Launching these applications will start a new interactive session with the previously submitted parameters.
+- ``sessions`` - the three most recent active interactive sessions
+- ``motd`` - the Message of the Day
+- ``xdmod_widget_job_efficiency`` - the XDMoD widget for job efficiency
+- ``xdmod_widget_jobs`` - the XDMoD widget for job information
+
+This feature also allows for administrators to *add* custom widgets.
+Simply drop new files into ``/etc/ood/config/apps/dashboard/views/widgets`` and reference them
+in the configuration. These partial files can be any format Rails recognizes, notably ``.html`` or
+``.html.erb`` extensions.
+
+Also if you use subdirectories under widgets, they can be referenced by relative paths. For example
+``views/widgets/cluster/_my_cluster_widget.html.erb`` would be referenced in the configuration
+as ``cluster/my_cluster_widget``.
+
+.. warning::
+
+ Rails expects files to be prefixed with an underscore. For example if you configured ``my_new_widget``
+ the filename should be ``_my_new_widget.html``.
+
+Without setting this configuration, the dashboard will arrange itself depending on what features are
+enabled. For example if both pinned apps and XDMoD features are enabled it will arrange itself accordingly
+based on a default layout.
+
+Here's the default configuration when all of these features are enabled.
+
+.. code:: yaml
+
+ # /etc/ood/config/ondemand.d/ondemand.yml
+ dashboard_layout:
+ rows:
+ - columns:
+ - width: 8
+ widgets:
+ - pinned_apps
+ - motd
+ - width: 4
+ widgets:
+ - xdmod_widget_job_efficiency
+ - xdmod_widget_jobs
+
+``rows`` are an array of row elements. Each row element has a ``columns`` field which is an array
+column elements. Each column element two fields. A ``width`` field that specifies the width in the
+`bootstrap grid layout`_ which defaults to 12 columns in total. It also has a ``widgets`` field which
+is an array of existing or newly added widgets to render in that column.
+
+.. _bootstrap grid layout: https://getbootstrap.com/docs/4.0/layout/grid/
+
+.. _customization_localization:
+
+Customize Text in OnDemand
+--------------------------
+
+Using Rails support for Internationalization (i18n), we have internationalized many strings in the Dashboard and the Job Composer apps.
+
+Initial translation dictionary files with defaults that work well for OSC and using the English locale (``en``) have been added (``/var/www/ood/apps/sys/dashboard/config/locales/en.yml`` and ``/var/www/ood/apps/sys/myjobs/config/locales/en.yml``). Sites wishing to modify these strings in order to provide site specific replacements for English, or use a different locale altogether, should do the following:
+
+#. Copy the translation dictionary file (or create a new file with the same structure of the keys you want to modify) to ``/etc/ood/config/locales/en.yml`` and modify that copy.
+#. If you want apps to look for these dictionary files in a different location than ``/etc/ood/config/locales/en.yml`` you can change the location by defining ``OOD_LOCALES_ROOT`` environment variable.
+#. The default locale is "en". You can use a custom locale. For example, if you want the locale to be French, you can create a ``/etc/ood/config/locales/fr.yml`` and then configure the Dashboard to use this locale by setting the environment variable ``OOD_LOCALE=fr`` where the locale is just the name of the file without the extension. Do this in either the nginx_stage config or in the Dashboard and Job Composer env config file.
+
+In each default translation dictionary file the values that are most site-specific (and thus relevant for change) appear at the top.
+
+.. list-table:: OnDemand Locale Files
+ :header-rows: 1
+ :stub-columns: 1
+
+ * - File path
+ - App
+ - Translation namespace
+ * - ``/var/www/ood/apps/sys/dashboard/config/locales/en.yml``
+ - `Dashboard`_
+ - ``dashboard``
+ * - ``/var/www/ood/apps/sys/myjobs/config/locales/en.yml``
+ - `Job Composer`_
+ - ``jobcomposer``
+ * - ``/etc/ood/config/locales/en.yml``
+ - All localizable apps will check this path, unless ``OOD_LOCALES_ROOT`` is set.
+ - Any
+
+.. warning::
+
+ Translations have certain variables passed to them for example ``%{support_url}``. Those variables may be used or removed from the translation. Attempting to use a variable that is not available to the translation will crash the application.
+
+.. note::
+
+ Localization files are YAML documents; remember that YAML uses spaces for indentation NOT tabs per the `YAML spec`_.
+
+.. note::
+
+ OnDemand uses the convention that translations that accept HTML with be suffixed with ``_html``. Any other translation will be displayed as plain text.
+
+.. Links for the OnDemand 1.7.0 release versions of these apps
+.. _Dashboard: https://github.com/OSC/ondemand/blob/master/apps/dashboard/config/locales/en.yml
+.. _Job Composer: https://github.com/OSC/ondemand/blob/master/apps/myjobs/config/locales/en.yml
+
+.. _Yaml spec: https://yaml.org/spec/1.2/spec.html#id2777534
+
+Change the Dashboard Tagline
+............................
+
+.. code-block:: yaml
+
+ en:
+ dashboard:
+ welcome_html: |
+ %{logo_img_tag}
+ OnDemand provides an integrated, single access point for all of your HPC resources.
+ motd_title: "Message of the Day"
+
+The ``welcome_html`` interpolates the variable ``logo_img_tag`` with the default
+logo, or the logo specified by the environment variable ``OOD_DASHBOARD_LOGO``.
+
+You may omit this variable in the value you specify for ``welcome_html`` if you prefer.
+
+Change quota messages in the Dashboard
+.......................................
+
+Two messages related to file system usage that sites may want to change:
+
+ - ``quota_additional_message`` - gives the user advice on what to do if they see a quota warning
+ - ``quota_reload_message`` - tells the user that they should reload the page to see their quota usage change, and by default also tells users that the quota values are updated every 5 minutes
+
+Customize Text in the Job Composer's options form
+.................................................
+
+The OSC-default value for ``options_account_help`` says that the account field is optional unless a user is a member of multiple projects.
+
+Items of note include what to call Accounts which might also be Charge Codes, or Projects. At OSC entering an account is optional unless a user is a member of multiple projects which is reflected in the default value for the string ``options_account_help``.
+
+Disk Quota Warnings on Dashboard
+--------------------------------
+
+You can display warnings to users on the Dashboard if their
+disk quota is nearing its limit. This requires an auto-updated (it is
+recommended to update this file every **5 minutes** with a cronjob) JSON file
+that lists all user quotas. The JSON schema for version `1` is given as:
+
+.. code:: json
+
+ {
+ "version": 1,
+ "timestamp": 1525361263,
+ "quotas": [
+ {},
+ {}
+ ]
+ }
+
+Where ``version`` defines the version of the JSON schema used, ``timestamp``
+defines when this file was generated, and ``quotas`` is a list of quota objects
+(see below).
+
+You can configure the Dashboard to use this JSON file (or files) by setting the
+environment variable ``OOD_QUOTA_PATH`` as a colon-delimited list of all JSON
+file paths in the ``/etc/ood/config/apps/dashboard/env`` file. In addition to
+pointing to files ``OOD_QUOTA_PATH`` may also contain HTTP(s) or FTP protocol
+URLs. Colons used in URLs are correctly handled and are not treated as delimiters.
+
+.. warning::
+
+ Sites using HTTP(s) or FTP for their quota files may see slower dashboard load
+ times, depending on the responsiveness of the server providing the quota file(s).
+
+The default threshold for displaying the warning is at 95% (`0.95`), but this
+can be changed with the environment variable ``OOD_QUOTA_THRESHOLD``.
+
+An example is given as:
+
+.. code:: sh
+
+ # /etc/ood/config/apps/dashboard/env
+
+ OOD_QUOTA_PATH="/path/to/quota1.json:https://example.com/quota2.json"
+ OOD_QUOTA_THRESHOLD="0.80"
+
+
+Individual User Quota
+.....................
+
+If the quota is defined as a ``user`` quota, then it applies to only disk
+resources used by the user alone. This is the default type of quota object and
+is given in the following format:
+
+.. code:: json
+
+ {
+ "type": "user",
+ "block_limit": 5000000,
+ "file_limit": 1000000,
+ "path": "/path/to/volume2",
+ "total_block_usage": 400000,
+ "total_file_usage": 1000,
+ "user": "user1"
+ }
+
+
+.. warning:: A block must be equal to 1 KiB for proper conversions.
+
+
+Individual Fileset Quota
+........................
+
+If the quota is defined as a ``fileset`` quota, then it applies to all disk
+resources used underneath a given volume. This requires the object to be
+repeated for **each user** that uses disk resources under this given volume.
+The format is given as:
+
+.. code:: json
+
+ {
+ "type": "fileset",
+ "user": "user1",
+ "path": "/path/to/volume2",
+ "block_usage": 500,
+ "total_block_usage": 1000,
+ "block_limit": 2000,
+ "file_usage": 1,
+ "total_file_usage": 5,
+ "file_limit": 10
+ }
+
+Where ``block_usage`` and ``file_usage`` are the disk resource usages attributed to
+the specified user only.
+
+.. note:: For each user with resources under this fileset, the above object will be repeated with just ``user``, ``block_usage``, and ``file_usage`` changing.
+
+
+.. _balance-warnings-on-dashboard:
+
+Balance Warnings on Dashboard
+--------------------------------
+
+You can display warnings to users on the Dashboard if their
+resource balance is nearing its limit. This requires an auto-updated (it is
+recommended to update this file daily with a cronjob) JSON file
+that lists all user balances. The JSON schema for version `1` is given as:
+
+.. code:: json
+
+ {
+ "version": 1,
+ "timestamp": 1525361263,
+ "config": {
+ "unit": "RU",
+ "project_type": "project"
+ },
+ "balances": [
+ {},
+ {}
+ ]
+ }
+
+Where ``version`` defines the version of the JSON schema used, ``timestamp``
+defines when this file was generated, and ``balances`` is a list of quota objects
+(see below).
+
+The value for ``config.unit`` defines the type of units for balances and
+``config.project_type`` would be project, account, or group, etc.
+Both values are used in locales and can be any string value.
+
+You can configure the Dashboard to use this JSON file (or files) by setting the
+environment variable ``OOD_BALANCE_PATH`` as a colon-delimited list of all JSON
+file paths.
+
+.. warning::
+
+ Sites using HTTP(s) or FTP for their balance files may see slower dashboard load
+ times, depending on the responsiveness of the server providing the quota file(s).
+
+The default threshold for displaying the warning is at ``0``, but this
+can be changed with the environment variable ``OOD_BALANCE_THRESHOLD``.
+
+An example is given as:
+
+.. code:: sh
+
+ # /etc/ood/config/apps/dashboard/env
+
+ OOD_BALANCE_PATH="/path/to/balance1.json:/path/to/balance2.json"
+ OOD_BALANCE_THRESHOLD=1000
+
+User Balance
+............
+
+If the balance is defined as a ``user`` balance, then it applies to only that user. Omit the ``project`` key:
+
+.. code:: json
+
+ {
+ "user": "user1",
+ "value": 10
+ }
+
+Project Balance
+...............
+
+If the balance is defined as a ``project`` balance, then it applies to a project/account/group, whatever is defined for ``config.project_type``:
+
+.. code:: json
+
+ {
+ "user": "user1",
+ "project": "project1",
+ "value": 10
+ }
+
+
+.. _maintenance-mode:
+
+Maintenance Mode
+-----------------
+
+
+As an administrator you may want to have some downtime of the Open OnDemand service for various reasons,
+while still telling your customers that the downtime is expected.
+
+You can do this by setting Open OnDemand in 'Maintenance Mode'.
+While Maintenance mode is active, Apache will not serve requests for paths outside the
+``/public/maintenance/*`` wildcard. Instead, it will serve the ``/var/www/ood/public/maintenance/index.html``
+file, which you can change or brand to be your own. Changes to this file will persist through upgrades.
+Any assets (e.g., images, stylesheets, or javascript) needed by the HTML file should be placed
+in the ``/var/www/ood/public/maintenance/`` directory. You can also put symbolic links into the
+``/var/www/ood/public/maintenance/`` directory, if you want to reuse assets located elsewhere in your
+file system.
+
+While in maintenance mode, Apache returns the HTML file and a 503 response code to all users whose
+IP does not match one of the configured allowlist regular expressions.
+The allowlist is to allow staff, localhost or a subset of your users access while restricting others.
+
+In this example we allow access to anyone from ``192.168.1..*`` which is the 192.168.1.0/24 CIDR and
+the single IP '10.0.0.1'.
+
+These are the settings you'll need for this functionality.
+
+.. code:: yaml
+
+ # /etc/ood/config/ood_portal.yml
+ use_rewrites: true
+ use_maintenance: true
+ maintenance_ip_allowlist:
+ # examples only! Your ip regular expressions will be specific to your site.
+ - '192.168.1..*'
+ - '10.0.0.1'
+
+To start maintenance mode (and thus start serving this page) simply ``touch /etc/ood/maintenance.enable``
+to create the necessary file. When your downtime is complete just remove the file and all the
+traffic will be served normally again. The existence of this file is what starts or stops maintenance
+mode, not it's content, so you will not need to restart apache or modify it's config files for this to
+take affect.
+
+
+.. _grafana-support:
+
+Grafana support
+---------------
+
+It's possible to display Grafana graphs within the ActiveJobs app when a user expands a given job.
+
+Grafana must be configured to support embedded panels and at this time it is also required to have an anonymous organization. Below are configuration options are needed to support displaying Grafana panels in ActiveJobs. Adjust `org_name` to match whatever organization you wish to be anonymous.
+
+.. warning::
+
+ Changing a Grafana install to support anonymous access can cause unintended consequences for how authenticated users interact with Grafana.
+ It's recommended to test anonymous access on a non-production Grafana install if you do not already support anonymous access.
+
+.. code:: shell
+
+ [auth.anonymous]
+ enabled = true
+ org_name = Public
+ org_role = Viewer
+
+ [security]
+ allow_embedding = true
+
+The dashboard used by OSC is the `OnDemand Clusters `_ dashboard.
+
+Settings used to access Grafana are configured in the cluster config. The following is an example from OSC:
+
+.. code:: yaml
+
+ custom:
+ grafana:
+ host: "https://grafana.osc.edu"
+ orgId: 3
+ dashboard:
+ name: "ondemand-clusters"
+ uid: "aaba6Ahbauquag"
+ panels:
+ cpu: 20
+ memory: 24
+ labels:
+ cluster: "cluster"
+ host: "host"
+ jobid: "jobid"
+ cluster_override: "mysite"
+
+When viewing a dashboard in Grafana choose the panel you'd wish to display and select `Share`.
+Then choose the `Embed` tab which will provide you with the iframe URL that will need to be generated within OnDemand.
+The time ranges and values for labels (eg: `var-cluster=`) will be autofilled by OnDemand.
+
+* ``orgId`` is the ``orgId`` query parameter
+* The dashboard ``name`` is the last segment of the URI before query parameters
+* The ``uid`` is the UID portion of URL that is unique to every dashboard
+* The ``panelId`` query parameter will be used as the value for either ``cpu`` or ``memory`` depending on the panel you have selected
+* The values for ``labels`` are how OnDemand maps labels in Grafana to values expected in OnDemand. The ``jobid`` key is optional, the others are required.
+* The ``cluster_override`` can override the cluster name used to make requests to Grafana if the Grafana cluster name varies from OnDemand cluster name.
+
+.. _disable-host-link-batch-connect:
+
+Disable Host Link in Batch Connect Session Card
+-----------------------------------------------
+
+Batch connect session cards like this have links to the compute node on which the job is currently running (highlighted).
+
+.. figure:: /images/bc-card-w-hostlink.png
+ :align: center
+
+However, some sites may want to disable this feature because they do not allow ssh sessions on the compute
+nodes.
+
+To disable this, simply set the environment variable in the dashboards' env file
+``/etc/ood/config/apps/dashboard/env`` to a falsy value (0, false, off).
+
+.. code:: sh
+
+ # don't show ssh link in batch connect card
+ OOD_BC_SSH_TO_COMPUTE_NODE=off
+
+If you wish to disable on a per-cluster basis, you can set the following in your :ref:`cluster YAML configuration `.
+
+.. code-block:: yaml
+ :emphasize-lines: 3-
+
+ v2:
+ # ...
+ batch_connect:
+ ssh_allow: false
+
+.. _set-illegal-job-name-characters:
+
+Set Illegal Job Name Characters
+-------------------------------
+
+If you encounter an issue in running batch connect applications complaining about invalid
+job names like the error below.
+
+``Unable to read script file because of error: ERROR! argument to -N option must not contain /``
+
+To resolve this set ``OOD_JOB_NAME_ILLEGAL_CHARS`` to ``/`` for all OOD applications in the
+``pun_custom_env`` attribute of the ``/etc/ood/config/nginx_stage.yml`` file.
+
+.. code-block:: yaml
+
+ # /etc/ood/config/nginx_stage.yml
+ pun_custom_env:
+ OOD_JOB_NAME_ILLEGAL_CHARS: "/"
+
+.. _customize_dex_theme:
+
+Customize Dex Theme
+-------------------
+
+It's possible to use a customized theme when authenticating with Dex when using OnDemand's default authentication.
+Refer to the upstream `Dex template docs`_ for additional information on templating Dex.
+
+The simplest approach is to copy the OnDemand theme and make changes. This is idea if you wish to make the following changes:
+
+- Change navigation or login page logos
+- Change favicon
+- Change CSS styles
+
+.. code-block:: sh
+
+ cp -r /usr/share/ondemand-dex/web/themes/ondemand /usr/share/ondemand-dex/web/themes/mycenter
+
+To update the theme you must modify ``/etc/ood/config/ood_portal.yml`` and regenerate the Dex configuration:
+
+.. code-block:: yaml
+ :emphasize-lines: 3-
+
+ dex:
+ # ...
+ frontend:
+ theme: mycenter
+
+The default ``ondemand`` theme can also be configured using the following configuration keys within ``/etc/ood/config/ood_portal.yml``:
+
+.. code-block:: yaml
+ :emphasize-lines: 4-
+
+ dex:
+ # ...
+ frontend:
+ issuer: "MyCenter OnDemand"
+ extra:
+ navLogo: "/public/nav-logo.png"
+ loginLogo: "/public/logo.png"
+ loginTitle: "Log in with your Center username and password"
+ loginButtonText: "Log in with your Center account"
+ usernamePlaceholder: "center-username"
+ passwordPlaceholder: "center-password"
+ loginAlertMessage: "Login services will be down during center maintenance between 8:00 AM EST and 10:00 AM EST"
+ loginAlertType: "warning"
+
+Changes are applied by running ``update_ood_portal`` and restarting the ``ondemand-dex`` service.
+
+.. code-block:: sh
+
+ sudo /opt/ood/ood-portal-generator/sbin/update_ood_portal
+ sudo systemctl restart ondemand-dex.service
+
+.. _dex template docs: https://dexidp.io/docs/templates/
+
+.. _xdmod_integration:
+
+XDMoD Integration
+-----------------
+
+
+XDMoD Integration requires XDMoD 9+, OnDemand 1.8+, and the ability to facilitate single sign on between the two services. Currently this has been demonstrated to work using OpenID Connect via Keycloak as well as a modified instance of Dex Identity Provider to support sessions.
+
+.. figure:: /images/customization_xdmod.png
+ :align: center
+
+ Example of XDMoD Job Efficiency reports in the OnDemand Dashboard.
+
+Steps to enable the XDMoD reports in the OnDemand Dashboard:
+
+#. Configure OnDemand with XDMoD host URL in PUN /etc/ood/config/nginx_stage.yml
+
+ .. code-block:: yaml
+
+ pun_custom_env:
+ OOD_XDMOD_HOST: "https://xdmod.osc.edu"
+
+#. Add OnDemand host as domain to XDMoD portal settings for CORS /etc/xdmod/portal_settings.ini
+
+ .. code-block:: none
+
+ domains = "https://ondemand.osc.edu"
+
+#. Configure identity provider to include OnDemand host in HTTP `Content-Security-Policy for frame-ancestors `_ since OnDemand uses iFrames to trigger SSO with XDMoD when a user logs in. Below is what we ensured Content-Security-Policy header for frame-ancestors was set to when configuring Keycloak:
+
+ .. code-block:: none
+
+ frame-ancestors https://*.osc.edu 'self'
+
+#. If you want the XDMoD links in the OnDemand Job Composer you also need to configure OnDemand with XDMoD resource id in each cluster config. For example, in the hpctoolset the resource_id for the hpc cluster is 1 in XDMoD, so we modify /etc/ood/config/clusters.d/hpc.yml to add a xdmod map to the custom map at the bottom of the file:
+
+ .. code-block:: yaml
+ :emphasize-lines: 10-
+
+ v2:
+ metadata:
+ title: "HPC Cluster"
+ login:
+ host: "frontend"
+ job:
+ adapter: "slurm"
+ cluster: "hpc"
+ bin: "/usr/bin"
+ custom:
+ xdmod:
+ resource_id: 1
+
+#. In the Job Composer, Open XDMoD job links will include a warning message that the job may not appear in XDMoD for up to 24 hours after the job completed. The message is to address the gap of time between the job appearing as completed in the Job Composer and the job appearing in Open XDMoD after the ingest and aggregation script is run. This message appears from the time the Job Composer becomes aware of the job completion status, till an elapsed time specified in seconds by the locale key ``en.jobcomposer.xdmod_url_warning_message_seconds_after_job_completion`` which defaults to 24 hours (86400 seconds) with a text message specified by locale key ``en.jobcomposer.xdmod_url_warning_message``. To disable this message, set the value you your locale file under ``/etc/ood/config/locales``. For example, in the default locale we have these values:
+
+ .. code-block:: yaml
+
+ en:
+ jobcomposer:
+ xdmod_url_warning_message: "This job may not appear in Open XDMoD until 24 hours after the completion of the job."
+ xdmod_url_warning_message_seconds_after_job_completion: 86400
+
+
+ Which results in these warning messages appearing in Job Composer:
+
+ .. figure:: /images/customization_xdmod_jobcomposer_warning_1.png
+ :align: center
+ .. figure:: /images/customization_xdmod_jobcomposer_warning_2.png
+ :align: center
+
+
+.. _remote-file-systems:
+
+Accessing Remote File Systems
+-----------------------------
+
+Since 2.1 you can use ``rclone`` to interact with remote file systems. Since
+every command in Open OnDemand is issued *as the user*, the user themselves
+are required to setup their ``rclone`` remotes.
+
+You can refer to the `OSC's rclone documentation`_ on how to configure rclone
+remotes.
+
+To enable this feature ensure that ``rclone`` is installed on the same machine
+that Open OnDemand is installed. You also have to enable the feature through
+the :ref:`configuration entry for enabling remote filesystems `.
+
+Cancel Interactive Sessions
+---------------------------
+
+We can now cancel an interactive session from the session panel without deleting the session card.
+This functionality will allow users to remove the job from the scheduler and keep the information in the OnDemand interface.
+
+This feature is disabled behind a feature toggle. To enable it, set the configuration property ``cancel_session_enabled: true``.
+For more information on how to configure properties, see :ref:`configuration documentation `.
+
+When enabled, the cancel button will appear for active sessions.
+When the session is cancelled, the job will be cancelled in the scheduler,
+the status will change to ``completed``, and the session card will be kept.
+For completed sessions, the system will only show the delete button.
+
+.. figure:: /images/cancel_session.png
+ :align: center
+
+.. include:: customizations/custom-pages.inc
+.. include:: customizations/support-ticket.inc
+
+.. _OSC's rclone documentation: https://www.osc.edu/resources/getting_started/howto/howto_use_rclone_to_upload_data
+.. _2.0 documentation for controlling the navbar: https://osc.github.io/ood-documentation/release-2.0/customization.html#control-which-apps-appear-in-the-dashboard-navbar
diff --git a/downcase-ids/_sources/enable-desktops.rst.txt b/downcase-ids/_sources/enable-desktops.rst.txt
new file mode 100644
index 000000000..91e4ec872
--- /dev/null
+++ b/downcase-ids/_sources/enable-desktops.rst.txt
@@ -0,0 +1,28 @@
+.. _enable-desktops:
+
+Enable Interactive Desktop
+==========================
+
+This installation guide will walk you through setting up an Interactive Desktop
+app that your users will be able to use to launch a Gnome 2, Mate, or Xfce
+desktop on a compute node within your HPC cluster. The user should then be able
+to connect to a running session through their browser using the `noVNC`_
+client.
+
+.. danger::
+
+ Confirm that you have walked through the
+ :ref:`app-development-interactive-setup` instructions for interactive apps
+ before continuing on.
+
+.. toctree::
+ :maxdepth: 2
+ :numbered: 1
+ :caption: Quick Start Guide
+
+ enable-desktops/software-requirements
+ enable-desktops/add-cluster
+ enable-desktops/modify-form-attributes
+ enable-desktops/custom-job-submission
+
+.. _novnc: http://novnc.com/info.html
diff --git a/downcase-ids/_sources/enable-desktops/add-cluster.rst.txt b/downcase-ids/_sources/enable-desktops/add-cluster.rst.txt
new file mode 100644
index 000000000..e49d35344
--- /dev/null
+++ b/downcase-ids/_sources/enable-desktops/add-cluster.rst.txt
@@ -0,0 +1,53 @@
+.. _enable-desktops-add-cluster:
+
+Add a Cluster
+=============
+
+We now need to add this cluster as a Desktop option in the Interactive Apps
+list. All customization is done underneath the root directory
+:file:`/etc/ood/config/apps/bc_desktop` which requires escalated privileges to
+modify.
+
+#. Start by creating the working directory:
+
+ .. code-block:: sh
+
+ mkdir -p /etc/ood/config/apps/bc_desktop
+
+#. For *each* cluster that we want to launch a Desktop session on we will need
+ a corresponding :ref:`app-development-interactive-form` configuration file
+ in YAML format located in this directory.
+
+ Assuming we want to launch a desktop on OSC's Oakley cluster we would have:
+
+ .. code-block:: yaml
+
+ # /etc/ood/config/apps/bc_desktop/oakley.yml
+ ---
+ title: "Oakley Desktop"
+ cluster: "oakley"
+
+ .. warning::
+
+ The ``cluster`` attribute above **MUST** match a valid cluster
+ configuration file located underneath
+ :file:`/etc/ood/config/clusters.d/`.
+
+#. Navigate to your OnDemand site, in particular the Dashboard App, and you
+ should see in the top dropdown menu "Interactive Apps" ⇒ "Oakley Desktop"
+ (or whatever you set as the ``title``).
+
+ After choosing "Oakley Desktop" from the menu, you should be presented with
+ a form to "Launch" a Desktop session to the given cluster.
+
+ Most likely if you click "Launch" it will fail miserably. That is because we
+ will need to configure the submission parameters for cluster's resource
+ manager.
+
+ .. note::
+
+ If by some chance when you click "Launch" and it actually successfully
+ submits a job to your cluster, it is **highly** recommended that you
+ click the link under "Session ID:". This will open the file browser
+ underneath the working directory of the batch job. This will allow you to
+ read all the logs generated to help debug any issues that may crop up.
diff --git a/downcase-ids/_sources/enable-desktops/custom-job-submission.rst.txt b/downcase-ids/_sources/enable-desktops/custom-job-submission.rst.txt
new file mode 100644
index 000000000..afd03b120
--- /dev/null
+++ b/downcase-ids/_sources/enable-desktops/custom-job-submission.rst.txt
@@ -0,0 +1,239 @@
+.. _enable-desktops-custom-job-submission:
+
+Custom Job Submission
+=====================
+
+The :ref:`app-development-interactive-submit` configuration file describes how
+the batch job should be submitted to your cluster. The location of this file
+**must** be specified in the respective
+:file:`/etc/ood/config/apps/bc_desktop/{my_cluster}.yml` form configuration
+file, so that when a user submits the form, the specified submission
+configuration is used when submitting the batch job.
+
+To customize job submission we will need to first edit our custom desktop app
+:ref:`app-development-interactive-form` YAML file as such:
+
+.. code-block:: yaml
+ :emphasize-lines: 5-
+
+ # /etc/ood/config/apps/bc_desktop/my_cluster.yml
+ ---
+ title: "My Cluster Desktop"
+ cluster: "my_cluster"
+ submit: "submit/my_submit.yml.erb"
+
+Notice we included the configuration option ``submit`` that points to our
+custom :ref:`app-development-interactive-submit` YAML configuration file. This
+can be an absolute file path or a relative file path with respect to the
+:file:`/etc/ood/config/apps/bc_desktop/` directory. It is important to notice
+you must use this or some other directory outside the app's root.
+
+.. note::
+
+ The ``*.erb`` file extension will cause the YAML configuration file to be
+ processed using the `eRuby (Embedded Ruby)`_ templating system. This allows
+ you to embed Ruby code into the YAML configuration file for flow control,
+ variable substitution, and more.
+
+.. danger::
+
+ Do not put the :ref:`app-development-interactive-submit` configuration file
+ directly underneath :file:`/etc/ood/config/apps/bc_desktop`. If you do, OOD will think
+ this a different app's ``form.yml``. Instead we typically
+ create the directory :file:`submit/` underneath the app's root directory and put our
+ :ref:`app-development-interactive-submit` configuration files underneath
+ that.
+
+.. _eruby (embedded ruby): https://en.wikipedia.org/wiki/ERuby
+
+We can now create and modify the :ref:`app-development-interactive-submit`
+configuration file at::
+
+ /etc/ood/config/apps/bc_desktop/submit/my_submit.yml.erb
+
+Since it has the extension ``.erb`` we can take advantage of the Ruby language
+to make the configuration file dynamic. In particular, you will now have access
+to the user-submitted form arguments defined as:
+
+bc_num_hours
+ *Default:* ``"1"``
+
+ A Ruby ``String`` containing the number of hours a user requested for the
+ Desktop batch job to run.
+
+bc_num_slots
+ *Default:* ``"1"``
+
+ A Ruby ``String`` containing either the number of nodes or processors
+ (depending on the type of resource manager the cluster uses) a user
+ requested.
+
+bc_account
+ *Default:* ``""``
+
+ A Ruby ``String`` that holds the account the user supplied to charge the job
+ against.
+
+bc_queue
+ *Default:* ``""``
+
+ A Ruby ``String`` that holds the queue the user requested for the job to run
+ on.
+
+bc_email_on_started
+ *Default:* ``"0"``
+
+ A Ruby ``String`` that can either be ``"0"`` (do not send the user an email
+ when the job starts) or ``"1"`` (send an email to the user when the job
+ starts).
+
+node_type
+ *Default:* ``""``
+
+ A Ruby ``String`` that can be used for more advanced job submission. This is
+ an advanced option that is disabled by default and does nothing if you do
+ enable it, unless you add it to a custom job submission configuration file.
+
+Some examples on how to submit jobs using the above form attributes are given
+in the following sections for the given resource manager.
+
+Slurm
+-----
+
+For most cases of Slurm you will want to modify how the ``bc_num_slots``
+(number of nodes) is submitted to the batch server.
+
+This can be handled in your custom job submission configuration file as such:
+
+.. code-block:: yaml
+
+ # /etc/ood/config/apps/bc_desktop/submit/my_submit.yml.erb
+ ---
+ script:
+ native:
+ - "-N"
+ - "<%= bc_num_slots.blank? ? 1 : bc_num_slots.to_i %>"
+
+All `batch script options`_ are underneath the ``script`` configuration option.
+In particular since there is no option to modify number of nodes, we need to
+directly interact with the ``native`` command line arguments. This is specified
+as an array of :command:`sbatch` arguments.
+
+.. note::
+
+ It is recommended you use the corresponding `batch script options`_ before
+ using the ``native`` fallback.
+
+Torque
+------
+
+For most cases of Torque you will want to modify how the ``bc_num_slots``
+(number of nodes) is submitted to the batch server.
+
+This can be handled in your custom job submission configuration file as such:
+
+.. code-block:: yaml
+
+ # /etc/ood/config/apps/bc_desktop/submit/my_submit.yml.erb
+ ---
+ script:
+ native:
+ resources:
+ nodes: "<%= bc_num_slots.blank? ? 1 : bc_num_slots.to_i %>:ppn=28"
+
+All `batch script options`_ are underneath the ``script`` configuration option.
+In particular since there is no option to modify number of nodes, we need to
+directly interact with the ``native`` command line arguments.
+
+For more information on the available options for the ``native`` attribute
+when using Torque please see the `pbs-ruby documentation`_.
+
+.. note::
+
+ It is recommended you use the corresponding `batch script options`_ before
+ using the ``native`` fallback.
+
+PBS Professional
+----------------
+
+For most cases of PBS Professional you will want to modify how the
+``bc_num_slots`` (number of CPUs on a single node) is submitted to the batch
+server.
+
+This can be handled in your custom job submission configuration file as such:
+
+.. code-block:: yaml
+
+ # /etc/ood/config/apps/bc_desktop/submit/my_submit.yml.erb
+ ---
+ script:
+ native:
+ - "-l"
+ - "select=1:ncpus=<%= bc_num_slots.blank? ? 1 : bc_num_slots.to_i %>"
+
+All `batch script options`_ are underneath the ``script`` configuration option.
+In particular since there is no option to modify number of nodes/cpus, we need
+to directly interact with the ``native`` command line arguments. This is
+specified as an array of :command:`qsub` arguments.
+
+If you would like to mimic how Torque handles ``bc_num_slots`` (number of
+**nodes**), then we will first need to change the form label of
+``bc_num_slots`` that the user sees in the form. This can be done by modifying
+our Desktop app local YAML configuration file:
+
+.. code-block:: yaml
+ :emphasize-lines: 5-7
+
+ # /etc/ood/config/apps/bc_desktop/submit/my_submit.yml.erb
+ ---
+ title: "Cluster1 Desktop"
+ cluster: "cluster1"
+ attributes:
+ bc_num_slots:
+ label: "Number of nodes"
+ submit: "submit/my_submit.yml.erb"
+
+Now when we go to the Desktop app form in our browser it will have the new
+label "Number of nodes" instead of "Number of CPUs on a single node".
+
+Next we will need to handle how we submit the ``bc_num_slots`` since it means
+something different now. So now modify the job submission configuration file as
+such:
+
+.. code-block:: yaml
+
+ # /etc/ood/config/apps/bc_desktop/submit/my_submit.yml.erb
+ ---
+ script:
+ native:
+ - "-l"
+ - "select=<%= bc_num_slots.blank? ? 1 : bc_num_slots.to_i %>:ncpus=28"
+
+You can also append ``mem=...gb`` to the ``select=...`` statement if you'd
+like.
+
+.. note::
+
+ It is recommended you use the corresponding `batch script options`_ before
+ using the ``native`` fallback.
+
+.. _batch script options: http://www.rubydoc.info/gems/ood_core/OodCore/Job/Script
+.. _pbs-ruby documentation: http://www.rubydoc.info/gems/pbs/PBS/Batch#submit_script-instance_method
+
+LinuxHost Adapter
+--------------------
+
+If you're using the :ref:`resource-manager-linuxhost` you actually don't *need* a specialized
+submit.yml.erb. There is no need to specify resources like the other adapters above.
+
+You can however, use it to override the adapter's global fields for mount binding and specifying
+which container use.
+
+.. code-block:: yaml
+
+ # /etc/ood/config/apps/bc_desktop/submit/linuxhost_submit.yml.erb
+ ---
+ batch_connect:
+ native:
+ singularity_bindpath: /etc,/media,/mnt,/opt,/run,/srv,/usr,/var,/fs,/home
+ singularity_container: /usr/local/modules/netbeans/netbeans_2019.sif
diff --git a/downcase-ids/_sources/enable-desktops/modify-form-attributes.rst.txt b/downcase-ids/_sources/enable-desktops/modify-form-attributes.rst.txt
new file mode 100644
index 000000000..c878a40b1
--- /dev/null
+++ b/downcase-ids/_sources/enable-desktops/modify-form-attributes.rst.txt
@@ -0,0 +1,312 @@
+.. _enable-desktops-modify-form-attributes:
+
+Modify Form Attributes
+======================
+
+In some cases you may want to modify the form presented to the user as well as
+any other configurable options. Some examples:
+
+- Use an Xfce desktop instead of Mate desktop.
+- Remove the "Queue" form field as your scheduler will auto select the correct
+ queue.
+- Hard-code the "Number of nodes" to just 1, so that users can't launch
+ desktops with multiple nodes.
+- Change the label for the form field "Account" to "Project".
+- Add help text to a given form field.
+- Change default value for a form field.
+
+The :ref:`app-development-interactive-form` YAML configuration file is
+responsible for defining these form attributes and how they are presented to
+the user.
+
+For each configuration file underneath::
+
+ /etc/ood/config/apps/bc_desktop/
+
+a separate desktop app will be presented as an option to the user from the
+dashboard, with the simplest :ref:`app-development-interactive-form`
+configuration file for an Interactive Desktop app given as:
+
+.. code-block:: yaml
+
+ # /etc/ood/config/apps/bc_desktop/my_cluster.yml
+ ---
+ title: "My Cluster Desktop"
+ cluster: "my_cluster"
+
+Before we begin modifying form attributes. Let us first take a look at the
+default form definition located in the source file
+:file:`/var/www/ood/apps/sys/bc_desktop/form.yml` (**do not modify**):
+
+.. code-block:: yaml
+
+ # /var/www/ood/apps/sys/bc_desktop/form.yml
+ ---
+ attributes:
+ desktop: "mate"
+ bc_vnc_idle: 0
+ bc_vnc_resolution:
+ required: true
+ node_type: null
+ form:
+ - bc_vnc_idle
+ - desktop
+ - bc_num_hours
+ - bc_num_slots
+ - node_type
+ - bc_account
+ - bc_queue
+ - bc_vnc_resolution
+ - bc_email_on_started
+
+The ``attributes`` and ``form`` configuration options can all be overridden in
+our global YAML configuration file. But typically you will only modify the
+``attributes`` options.
+
+In the following sections you will find common examples on how to override the
+above options.
+
+.. warning::
+
+ The ``form`` configuration option defines all the available attributes as
+ well as the order they appear in the form (it is an array).
+
+ Caution must be taken if you decide to override the ``form`` configuration
+ option. As this is an array, you can't simply prepend or append, you will
+ need to completely redefine it with your included modifications.
+
+Change to Xfce Desktop
+----------------------
+
+The default installation has the ``desktop`` attribute hard-coded to the value
+``"mate"``. If you would like to change this to use ``"xfce"`` you can make the
+following edits to your custom YAML configuration file:
+
+.. code-block:: yaml
+ :emphasize-lines: 5-
+
+ # /etc/ood/config/apps/bc_desktop/my_cluster.yml
+ ---
+ title: "My Cluster Desktop"
+ cluster: "my_cluster"
+ attributes:
+ desktop: "xfce"
+
+And all Desktops will attempt to launch the Xfce desktop.
+
+.. note::
+
+ Whenever you hard-code a form attribute to a value like ``"xfce"`` in the
+ above case, no input field will appear in the form for the user to fill in.
+ So in the above case, the user cannot specify the ``desktop`` attribute in
+ the form because we hard-coded it.
+
+Remove Form Field
+-----------------
+
+To remove a form field such as "Queue" defined under the attribute ``bc_queue``
+from the Desktop form you can make the following edits to your custom YAML
+configuration file:
+
+.. code-block:: yaml
+ :emphasize-lines: 5-
+
+ # /etc/ood/config/apps/bc_desktop/my_cluster.yml
+ ---
+ title: "My Cluster Desktop"
+ cluster: "my_cluster"
+ attributes:
+ bc_queue: null
+
+After refreshing the form in your browser you should not see the "Queue" field
+anymore.
+
+Basically we are hard-coding the value of ``bc_queue`` to be the YAML type
+``null``. And as we discussed in the previous example whenever you hard-code an
+attribute, it will not show up in the form.
+
+.. warning::
+
+ If you have any
+ :ref:`enable-desktops-custom-job-submission` configuration files that use
+ this attribute, they will receive empty strings ``""``, so you will need to
+ test if they are blank before handling them.
+
+Hard-code a Form Field
+----------------------
+
+If we want to remove a form field but define its value to something other than
+a blank string, we can set the attribute's value directly.
+
+For example, if you don't want users to submit Desktops with more than 1 node
+under the attribute ``bc_num_slots``, you can make the following edits to your
+custom YAML configuration file:
+
+.. code-block:: yaml
+ :emphasize-lines: 5-
+
+ # /etc/ood/config/apps/bc_desktop/my_cluster.yml
+ ---
+ title: "My Cluster Desktop"
+ cluster: "my_cluster"
+ attributes:
+ bc_num_slots: 1
+
+As in the previous two examples, since we are hard-coding the value of the
+attribute, the form field will not show up and the user is unable to change
+this value. For the above case, the attribute ``bc_num_slots`` will always
+return ``"1"``.
+
+.. warning::
+
+ If you have any :ref:`enable-desktops-custom-job-submission` configuration
+ files that use this attribute, care must be taken when handling the
+ attribute as it will always come back as a `Ruby String`_.
+
+ So if you hard-coded an attribute to the integer ``1`` it will come back as
+ the string ``"1"`` and if you perform any arithmetic operations on this
+ attribute it will require you convert this back to an integer with the
+ method ``String#to_i``.
+
+Change a Label
+--------------
+
+You are able to modify the label for a corresponding attribute that appears
+above the input field in the form.
+
+For example, if you want to change the label for the "Account" form field given
+by the ``bc_account`` attribute to instead display "Project". This can be
+modified with the following edits to your custom YAML configuration file:
+
+.. code-block:: yaml
+ :emphasize-lines: 5-
+
+ # /etc/ood/config/apps/bc_desktop/my_cluster.yml
+ ---
+ title: "My Cluster Desktop"
+ cluster: "my_cluster"
+ attributes:
+ bc_account:
+ label: "Project"
+
+The key here is that we are defining a hash for the ``bc_account`` attribute
+instead of hard-coding it to a specific value. This means we will only override
+the equivalent option for this attribute (for the above example we are
+overriding the ``label`` option for the ``bc_account`` attribute).
+
+Now when you refresh the form in your browser, you should now see an input
+field with the label "Project".
+
+.. warning::
+
+ If you have any :ref:`enable-desktops-custom-job-submission` configuration
+ files that use this attribute, changing the label of the attribute will not
+ affect the value received by the user upon form submission.
+
+ But care must be taken that if by changing the label of the attribute you
+ also change the *meaning* of the attribute, then you may have to handle it
+ differently. For example, changing a label of "Number of processors" to
+ "Number of nodes" will have consequences on how you submit the job.
+
+Add Help Message to Field
+-------------------------
+
+You are also able to add a help message to any given form field through its
+corresponding attribute.
+
+For example, if you would like to add a help message to the attribute
+``bc_account`` you can make the following edits to your custom YAML
+configuration file:
+
+.. code-block:: yaml
+ :emphasize-lines: 5-
+
+ # /etc/ood/config/apps/bc_desktop/my_cluster.yml
+ ---
+ title: "My Cluster Desktop"
+ cluster: "my_cluster"
+ attributes:
+ bc_account:
+ help: "You can leave this blank if **not** in multiple projects."
+
+The key here is that we are defining a hash for the ``bc_account`` attribute
+instead of hard-coding it to a specific value. This means we will only override
+the equivalent option for this attribute (for the above example we are
+overriding the ``help`` option for the ``bc_account`` attribute).
+
+Now when you refresh the form in your browser, you should see the help message
+below the "Account" form input field.
+
+.. note::
+
+ Help messages can be written in Markdown_ format, but it is best not to get
+ carried away in the size of the help message.
+
+Change Field Default Value
+--------------------------
+
+You are able to modify the default value of a form field for a given attribute,
+which should not be confused with hard-coding a value for an attribute.
+
+For example, if you would like the form field "Number of hours" given by
+``bc_num_hours`` to be ``8`` hours by default, but still allow the user to
+change it then you can make the following edits in your custom YAML
+configuration file:
+
+.. code-block:: yaml
+ :emphasize-lines: 5-
+
+ # /etc/ood/config/apps/bc_desktop/my_cluster.yml
+ ---
+ title: "My Cluster Desktop"
+ cluster: "my_cluster"
+ attributes:
+ bc_num_hours:
+ value: 8
+
+The key here is that we are defining a hash for the ``bc_num_hours`` attribute
+instead of hard-coding it to a number. This means we want to override the
+equivalent option for this attribute (for the above example we are overriding
+the ``value`` option for the ``bc_num_hours`` attribute).
+
+Now when you refresh the desktop form in your browser, you should see a default
+value of ``8`` in the "Number of hours" form field.
+
+.. note::
+
+ There is a possibility you may see a number other than ``8`` in the above
+ example. That is because the Interactive Apps tool built into the Dashboard
+ **remembers** your last successful app launch for a corresponding app. So
+ when you go back to the form page for that given app, it will auto-fill in
+ the form with your previous values.
+
+
+Minimal LinuxHost Form
+----------------------
+
+Because the :ref:`resource-manager-linuxhost` is not like a traditional scheduler,
+there are very few form items you'll need to add or have your users choose from.
+
+This is a minimal configuration to launch an XFCE desktop environment on a cluster
+we call owens_login. You'll notice a lot of entries are null because they don't
+really have any meaning in the LinuxHost Adapter.
+
+.. code-block:: yaml
+ :emphasize-lines: 5-
+
+ # /etc/ood/config/apps/bc_desktop/owens_login_desktop.yml
+ ---
+ title: Owens Login XFCE desktop
+ description: This launches a XFCE desktop on an Owens login nodes.
+ cluster: owens_login
+ form:
+ - desktop
+ - bc_num_hours
+ attributes:
+ bc_num_hours:
+ value: 1
+ desktop: "xfce"
+
+
+.. _ruby string: https://ruby-doc.org/core-2.2.0/String.html
+.. _markdown: https://en.wikipedia.org/wiki/Markdown
diff --git a/downcase-ids/_sources/enable-desktops/software-requirements.rst.txt b/downcase-ids/_sources/enable-desktops/software-requirements.rst.txt
new file mode 100644
index 000000000..3aec1a68a
--- /dev/null
+++ b/downcase-ids/_sources/enable-desktops/software-requirements.rst.txt
@@ -0,0 +1,23 @@
+.. _enable-desktops-software-requirements:
+
+Software Requirements
+=====================
+
+The interactive Desktop app requires a Desktop Environment be installed on the
+nodes that the batch job is meant to run on, **NOT** the OnDemand node.
+
+The following desktops are currently supported:
+
+- `Xfce Desktop`_ 4+
+- `Mate Desktop`_ 1+ (**default**)
+- `Gnome Desktop`_ 2 (currently we do not support Gnome 3)
+
+.. warning::
+
+ Do **NOT** install the Desktop Environment on the **OnDemand node**. The
+ above Desktop Environments are **ONLY** for the **compute or login nodes** you intend
+ on launching Desktops on within batch jobs.
+
+.. _gnome desktop: https://www.gnome.org/
+.. _mate desktop: https://mate-desktop.org/
+.. _xfce desktop: https://xfce.org/
diff --git a/downcase-ids/_sources/glossary.rst.txt b/downcase-ids/_sources/glossary.rst.txt
new file mode 100644
index 000000000..2a2180464
--- /dev/null
+++ b/downcase-ids/_sources/glossary.rst.txt
@@ -0,0 +1,20 @@
+.. _glossary:
+
+Glossary
+========
+
+ Cluster
+ Physical machines with a resource scheduler that users can submit jobs to.
+
+ Compute-node
+ The machine where a submitted job runs. Part of a cluster.
+
+ Login-node
+ A server on the compute cluster that can be anything from the OOD dashboard itself, to a shell, to the file-browser app, etc.
+ Runs on the cluster but is not a compute-node.
+
+ Web-node
+ A term used for when a site or institution has enough funds/personel to run the OOD login on a dedicated server and not on a login-node.
+
+ PUN
+ The Per User Nginx. An Nginx instance running as the user.
diff --git a/downcase-ids/_sources/how-tos/analytics/google-analytics.rst.txt b/downcase-ids/_sources/how-tos/analytics/google-analytics.rst.txt
new file mode 100644
index 000000000..f33c25880
--- /dev/null
+++ b/downcase-ids/_sources/how-tos/analytics/google-analytics.rst.txt
@@ -0,0 +1,115 @@
+.. _google-analytics:
+
+Adding Google Analytics
+========================
+
+.. _Google service accounts: https://cloud.google.com/iam/docs/service-accounts
+.. _Google IAM roles: https://cloud.google.com/iam/docs/understanding-roles
+.. _Google Analytics: https://analytics.google.com/analytics/web
+
+If you wish you can setup your Open-OnDemand instance to send usage data to Google Analytics
+(GA) that you can then query and report on, this page walks through how to do just that.
+
+.. note::
+
+ You'll need to have a `Google Analytics`_ account setup as a prerequisite to this.
+
+ To query GA You'll need to have to have a service account setup with the appropriate permissions. See
+ info on `Google service accounts`_ and `Google IAM roles`_ for more details on that.
+
+
+Configure Open OnDemand
+--------------------------
+
+Refer to the :ref:`ondemand.d configuration property google_analytics_tag_id `
+on how to configure this feature.
+
+
+Querying Google Analytics
+---------------------------
+
+.. _GA client libraries: https://developers.google.com/analytics/devguides/reporting/core/v3/libraries
+
+.. warning::
+ This documentation is for GA version 3. With the newer versions of GA this may not
+ work as intended. As OSC does not use GA we're unable to update these examples
+ ourselves, but would accept updates for the same.
+
+Now that you have Open-OnDemand sending information to GA and it's all configured correctly,
+you can now query GA for this information, parse it and present it in any fashion you like.
+
+Here's a small portion of how we query GA in ruby, but there are many `GA client libraries`_
+available.
+
+This example is not complete and is only meant to illustrate how to query GA given the defined
+metric set above. Let's go through each of these things.
+
+.. code-block:: ruby
+
+ # Dimensions - here we want dimensions 1, 3 and something called pagePath which is the web
+ # page requested. pagePath is a google predefined dimension that we populated. Dimensions 1
+ # and 3 were created above and are the username and timestamp (this is why the order in
+ # which they're defined is important).
+ DIMENSIONS = %w(
+ ga:dimension3
+ ga:dimension1
+ ga:pagePath
+ )
+
+ # we only want to report the hit metrics
+ METRICS = %w(
+ ga:hits
+ )
+
+ # First we specify the host so that we only get metrics from a specific host. Secondly,
+ # we filter only only 200 responses (dimension6 is status code) and we don't want to
+ # report on file editor edits.
+ FILTERS = %W(
+ ga:hostname==#{HOST};ga:dimension6==200;ga:pagePath!=/pun/sys/file-editor/edit
+ )
+
+ # now we can create our analytics object and make the query
+ analytics = Google::Apis::AnalyticsV3::AnalyticsService.new
+
+ # Here we query for the data that we want. A lot of things are omitted in this example
+ # for brevity like START_DATE (dynamic query times like the first day of the month)
+ # or GA_PROFILE (part of our credentials). And the fact that this is in a loop paginating
+ # the results, updating 'start_index' and only requesting STEP_SIZE (10,000 in our case)
+ # results at a time.
+ results = analytics.get_ga_data(
+ "ga:#{GA_PROFILE}",
+ START_DATE,
+ END_DATE,
+ METRICS.join(','),
+ dimensions: DIMENSIONS.empty? ? nil : DIMENSIONS.join(','),
+ filters: FILTERS.empty? ? nil : FILTERS.join(','),
+ sort: SORT.empty? ? nil : SORT.join(','),
+ start_index: start_index,
+ max_results: STEP_SIZE
+ )
+
+ target = open('my-report', "w")
+
+ # now we can write out the results in a format that I want for my reporting.
+ results.rows.each do |row|
+ begin
+ app = row[2]
+ row[2] = parse_uri(app, user: row[1])
+ row << app
+ target.write "#{row.join('|')}\n"
+ end
+
+
+More Info
+-----------
+
+.. _GA measurement protocol: https://developers.google.com/analytics/devguides/collection/protocol/v1/reference
+.. _analytics lua code: https://github.com/OSC/ondemand/blob/master/mod_ood_proxy/lib/analytics.lua
+
+For reference, here's more detailed information about implementations and protocols described
+in this document.
+
+See our `analytics lua code`_ for the implementation of how we're extracting this information,
+parsing it and sending it to Google.
+
+See the `GA measurement protocol`_ for more details on the format we're sending this data in.
\ No newline at end of file
diff --git a/downcase-ids/_sources/how-tos/app-development.rst.txt b/downcase-ids/_sources/how-tos/app-development.rst.txt
new file mode 100644
index 000000000..c8be31e7c
--- /dev/null
+++ b/downcase-ids/_sources/how-tos/app-development.rst.txt
@@ -0,0 +1,24 @@
+.. _app-development:
+
+App Development
+===============
+
+OnDemand is made up of the platform (Apache and NGINX), Passenger apps and
+plugins. Passenger apps are rack based Ruby apps, wsgi based Python web apps, or
+Node.js apps that follow a convention for the app's startup file.
+
+The Dashboard app, Shell app, and all other core apps in OnDemand are Passenger
+apps that can be replaced by custom Passenger apps. Or you can create your own.
+
+OnDemand's Interactive Apps are plugins that contain configuration files and a job
+template for running a VNC Server or Web Server application (such as Jupyter or MATLAB)
+on a compute node.
+
+
+.. toctree::
+ :maxdepth: 2
+ :caption: App Development Guide
+
+ app-development/enabling-development-mode
+ app-development/interactive
+ app-development/app-sharing
diff --git a/downcase-ids/_sources/how-tos/app-development/app-sharing.rst.txt b/downcase-ids/_sources/how-tos/app-development/app-sharing.rst.txt
new file mode 100644
index 000000000..a82654a32
--- /dev/null
+++ b/downcase-ids/_sources/how-tos/app-development/app-sharing.rst.txt
@@ -0,0 +1,246 @@
+.. _app_sharing:
+
+App Sharing
+=============
+
+Overview
+--------
+
+Apps may be shared via a variety of methods including:
+
+1. System installed by admin for everyone to launch (access controlled via file
+ permissions)
+2. Source Code Sharing (share code, launch your own copy of an app). This is
+ like how the Job Composer works, or how many groups collaborate when using
+ HPC: they share their job scripts and code, and use their own copies to
+ submit jobs.
+3. Executable Sharing (peer to peer) is where a user deploys an app in their
+ home directory that other users can launch. This is similar to adding to your
+ PATH the bin directory of another user.
+
+.. _app_sharing_system_installed_apps:
+
+System Installed Apps
+---------------------
+
+Admins may install apps to the system by copying the application directory to ``/var/www/ood/apps/sys``. Default directory permissions (``755``) will allow all users with access to OnDemand to see and run that app. Apps may have their access restricted by changing the permissions on individual application directories. For example if a site does not wish to show licensed software to un-licensed users they might do the following:
+
+ .. code-block:: sh
+
+ # Given:
+ # - an app named $NEW_APP
+ # - a group named $NEW_APP_GROUP
+ # - and a user named $NEW_APP_USER
+
+ sudo cp -r "/path/to/$NEW_APP" /var/www/ood/apps/sys
+ sudo chmod 750 "/var/www/ood/apps/sys/$NEW_APP"
+ sudo chgrp "$NEW_APP_GROUP" "/var/www/ood/apps/sys/$NEW_APP"
+
+ sudo usermod -a -G "$NEW_APP_GROUP" "$NEW_APP_USER"
+
+You can utilize file access lists (FACLs) to increase the granularity with whom you share the apps. For example, you could specify multiple groups and individual users to share with, or even exclude specific users or groups.
+
+This app authorization mechanism (using file permissions) can also be useful for canary deployments (sharing an app under development with a subset of users, or just your development team).
+
+Example Using File Permissions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+ .. code-block:: sh
+ :emphasize-lines: 7
+
+ [root@ood sys]# pwd
+ /var/www/ood/apps/sys
+ [root@ood sys]# ls -l
+ total 20
+ drwxr-xr-x. 14 root root 4096 Jul 26 15:20 activejobs
+ drwxr-xr-x. 3 root root 169 Jul 26 15:20 bc_desktop
+ drwxr-xr-x. 5 root root 157 Aug 1 18:19 bc_desktop_example_kde
+ drwxr-xr-x. 13 root root 4096 Jul 26 15:20 dashboard
+ drwxr-xr-x. 14 root root 4096 Jul 26 15:20 file-editor
+ drwxr-xr-x. 7 root root 4096 Jul 26 15:20 files
+ drwxr-xr-x. 14 root root 4096 Jul 26 15:20 myjobs
+ drwxr-xr-x. 7 root root 245 Jul 26 15:20 shell
+
+ .. figure:: /images/app-sharing-permissions-before.png
+ :align: center
+
+To restrict usage to only members of the ``desktopers`` group:
+
+ .. code-block:: sh
+ :emphasize-lines: 9
+
+ [root@ood sys]# pwd
+ /var/www/ood/apps/sys
+ [root@ood sys]# chmod 750 bc_desktop_example_kde/
+ [root@ood sys]# chgrp desktopers bc_desktop_example_kde
+ [root@ood sys]# ls -l
+ total 20
+ drwxr-xr-x. 14 root root 4096 Jul 26 15:20 activejobs
+ drwxr-xr-x. 3 root root 169 Jul 26 15:20 bc_desktop
+ drwxr-x---. 5 root desktopers 157 Aug 1 18:19 bc_desktop_example_kde
+ drwxr-xr-x. 13 root root 4096 Jul 26 15:20 dashboard
+ drwxr-xr-x. 14 root root 4096 Jul 26 15:20 file-editor
+ drwxr-xr-x. 7 root root 4096 Jul 26 15:20 files
+ drwxr-xr-x. 14 root root 4096 Jul 26 15:20 myjobs
+ drwxr-xr-x. 7 root root 245 Jul 26 15:20 shell
+
+Note that user ``ood`` is not a member of the ``desktopers`` supplemental group.
+
+ .. figure:: /images/app-sharing-permissions-after.png
+ :align: center
+
+These changes take effect immediately, although when a user is added or removed from a group their PUN will need to be restarted for the change to take effect.
+
+
+Code Sharing
+------------
+
+Code sharing is when an application's source code is shared between two or more users who run it as a personal development application. Models for this sharing can include using a web-based file repository such as Github, emailing Zip'd app directories, or a group readable directory symlinked to each user's ``~/ondemand/dev/`` directory.
+
+For an example of the later consider:
+
+ .. code-block:: sh
+
+ # As user mrodgers
+ owens-login01:mrodgers mrodgers$ pwd
+ # /fs/project/PZS0714/mrodgers
+ owens-login01:mrodgers mrodgers$ ls -l
+ # total 97856
+ # drwxr-xr-x 7 mrodgers PZS0714 4096 Aug 1 16:03 blender-batch-render-app
+ owens-login01:mrodgers mrodgers$ cd ~/ondemand/dev
+ owens-login01:mrodgers mrodgers$ ln -s /fs/project/PZS0714/mrodgers/blender-batch-render-app
+
+ # As user johrstrom
+ owens-login01:johrstrom johrstrom$ cd ~/ondemand/dev
+ owens-login01:johrstrom johrstrom$ ln -s /fs/project/PZS0714/mrodgers/blender-batch-render-app
+
+User ``johrstrom`` will now see ``blender-batch-render-app`` in their Sandbox Apps, but because they do not own the files they will not be able to edit the files, or update dependencies, etc resulting in a slightly broken experience. Better still would be peer to peer app sharing.
+
+
+Peer to Peer Executable Sharing
+-------------------------------
+
+By setting a few environment variables it is possible to enable a more polished peer to peer app sharing experience. There are two reasons why this mode is not always enabled: the first is that app permissions are the only thing that prevents all a site's OnDemand users from seeing a shared app, so it is important to get the permissions correct, and only to deploy apps that are production ready. The other reason to be careful with app sharing is that requires greater trust placed in app developers.
+
+.. warning:: Executable sharing means the app and all its code runs as the user
+ executing it, like everything else in OnDemand. User's might not
+ realize this. We currently do not provide an opt in screen warning
+ users that this app "will have permission to do everything on their
+ behalf and act as them". As a result, you should fully trust whoever
+ you enable to do share apps using executable sharing.
+
+Enabling The App Sharing Dashboard
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+#. To enable App Sharing in the Dashboard, set ``OOD_APP_SHARING=1`` in
+ ``/etc/ood/config/apps/dashboard/env``.
+#. Set ``OOD_DASHBOARD_SUPPORT_EMAIL=your@email.edu`` to add a link to support
+ for finding an app.
+#. Set ``OOD_APP_CATALOG_URL=https://link.to.online/app/catalog`` to link
+ externally to an advertised listing of apps available.
+#. Pin usr apps to the dashboard and group them by category.
+
+.. code:: yaml
+
+ # /etc/ood/config/ondemand.d/ondemand.yml
+ pinned_apps:
+ - 'usr/*'
+
+ pinned_apps_group_by: 'category'
+
+
+Enabling App Sharing in the dashboard serves two primary purposes:
+
+1. For shared app users, provide an interface to launch those apps
+2. For app developers, provide an interface to help manage shared apps
+
+Currently this significantly changes the interface of the Dashboard. The MOTD
+moves to the right of the screen and shared apps appear below the welcome logo
+and text.
+
+Before:
+
+.. figure:: /images/app-sharing-mode-before.png
+ :align: center
+
+
+After:
+
+.. figure:: /images/app-sharing-mode-after.png
+ :align: center
+
+Controlling Who Can Share and Access Apps
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Shared apps are deployed to
+``/var/www/ood/apps/usr/$USER/gateway/$APPNAME``. We recommend ``gateway``
+be a symlink to the user's home directory at ``$HOME/ondemand/share`` and
+by default set ``750`` permissions on ``/var/www/ood/apps/usr/$USER``. This
+approach has these benefits (assuming users named ``efranz`` and ``an0047``:
+
+#. The admin as root controls who can share apps by creating root owned
+ directories like ``/var/www/ood/apps/usr/efranz`` and
+ ``/var/www/ood/apps/usr/an0047``.
+#. The admin controls who can access that user's shared apps by setting
+ permissions on this directory. Thus by setting ``750`` on
+ ``/var/www/ood/apps/usr/an0047`` this ensures that an0047 can only share
+ apps with users in his primary group. At times we have created a \
+ supplemental group (shinyusr) and chgrp the share directory to this group so
+ that the developer can share apps with every user in this group.
+#. The developer who can share apps can modify permissions on the app
+ directories themselves i.e.
+ ``/var/www/ood/apps/usr/an0047/gateway/customapp``
+ so that only a subset of the users he could share with have access. This can
+ be done through FACLs or using the same chgrp + 755 approach
+
+In summary, to enable a new user to create shared apps, run these commands:
+
+.. code:: sh
+
+ # given a user efranz
+ sudo mkdir -p /var/www/ood/apps/usr/efranz
+ cd /var/www/ood/apps/usr/efranz
+ chmod 750 .
+ ln -s ~efranz/ondemand/share gateway
+
+Example of Executable Sharing
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This is with two users Eric (efranz) and Bob (an0047).
+
+Eric has a dev app "MATLAB", and interactive plugin app. Eric can
+
+1. Launch MATLAB
+2. View and Edit the code
+
+.. figure:: /images/app-sharing-1.png
+ :align: center
+
+
+
+Bob (an0047) cannot see this app because it is isolated in Eric's "Sandbox"
+i.e. ``~efranz/ondemand/dev/matlab``:
+
+.. figure:: /images/app-sharing-2.png
+ :align: center
+
+If Eric shares the git repo path or URL with Bob, Bob can clone this into his
+home directory if he is enabled as a developer. This is called "Source Code Sharing".
+
+Eric can share this app with Bob by selecting "My Shared Apps" and cloning the MATLAB
+repo to deploy ``~efranz/ondemand/share/matlab``
+
+.. figure:: /images/app-sharing-3.png
+ :align: center
+
+.. figure:: /images/app-sharing-4.png
+ :align: center
+
+.. figure:: /images/app-sharing-5.png
+ :align: center
+
+Now when Bob accesses the OnDemand home page he sees Eric's MATLAB app and can
+launch it:
+
+.. figure:: /images/app-sharing-6.png
+ :align: center
diff --git a/downcase-ids/_sources/how-tos/app-development/enabling-development-mode.rst.txt b/downcase-ids/_sources/how-tos/app-development/enabling-development-mode.rst.txt
new file mode 100644
index 000000000..d5734b3c8
--- /dev/null
+++ b/downcase-ids/_sources/how-tos/app-development/enabling-development-mode.rst.txt
@@ -0,0 +1,118 @@
+.. _enabling-development-mode:
+
+Enabling App Development
+========================
+
+Enable in OnDemand v1.6+:
+.........................
+
+Here are example steps to enable a user "efranz", assuming efranz's home directory is at ``/home/efranz``:
+
+#. Create a symlink so OnDemand finds efranz's apps:
+
+ .. code:: sh
+
+ sudo mkdir -p /var/www/ood/apps/dev/efranz
+ cd /var/www/ood/apps/dev/efranz
+ sudo ln -s /home/efranz/ondemand/dev gateway
+
+
+#. Have efranz access the Dashboard, and efranz will see the Develop dropdown
+
+
+Enable in OnDemand v1.4 & v1.5:
+...............................
+
+Here are example steps to enable a user "efranz", assuming efranz's home directory is at ``/home/efranz``:
+
+#. Create a symlink so OnDemand finds efranz's apps:
+
+ .. code:: sh
+
+ sudo mkdir -p /var/www/ood/apps/dev/efranz
+ cd /var/www/ood/apps/dev/efranz
+ sudo ln -s /home/efranz/ondemand/dev gateway
+
+#. Create dev directory ``/home/efranz/ondemand/dev`` where efranz's dev apps will go (or ask efranz to do that)
+#. Have efranz access the Dashboard, and efranz will see the Develop dropdown
+
+
+Enable in OnDemand v1.3:
+........................
+
+Here are example steps to enable a user "efranz", assuming efranz's home directory is at ``/home/efranz``:
+
+#. Create dev directory ``/home/efranz/ondemand/dev`` where efranz's dev apps will go (or ask efranz to do that)
+#. Have efranz access the Dashboard, and efranz will see the Develop dropdown. (if this doesn't happen, )
+
+.. note::
+
+ The rest of the documentation below assumes you are working with OnDemand 1.4+.
+
+
+Specify dedicated host for development (optional)
+....................................................
+
+The default host for the shell app is typically a login node. If this node does not contain a similar environment as the host OnDemand is installed on, including access to Software Collection (SCL) packages, it may be useful to provide developers a dedicated development host they can SSH to. This could even be the OnDemand host itself. If you configure the OnDemand dashboard to know about the dedicated development host, the dashboard will present links to open the shell app to this host.
+
+To specify a dedicated development host so OnDemand, set ``OOD_DEV_SSH_HOST`` environment variable for the dashboard in the file ``/etc/ood/config/apps/dashboard/env``. For example at OSC one of our OnDemand installs uses ondemand-test.osc.edu for the development host so we have this line: `OOD_DEV_SSH_HOST="ondemand-test.osc.edu" `_
+
+
+Make everyone a developer by default (optional)
+...............................................
+
+To revert to the way developer enabling worked in OnDemand 1.3, change the nginx_stage app_root configuration for dev apps by modifying /etc/ood/config/nginx_stage.yml and replacing
+
+.. code-block:: yaml
+ :emphasize-lines: 2
+
+ app_root:
+ dev: '/var/www/ood/apps/dev/%{owner}/gateway/%{name}'
+ usr: '/var/www/ood/apps/usr/%{owner}/gateway/%{name}'
+ sys: '/var/www/ood/apps/sys/%{name}'
+
+
+with
+
+
+.. code-block:: yaml
+ :emphasize-lines: 2
+
+ app_root:
+ dev: '~%{owner}/%{portal}/dev/%{name}'
+ usr: '/var/www/ood/apps/usr/%{owner}/gateway/%{name}'
+ sys: '/var/www/ood/apps/sys/%{name}'
+
+Then users can just create the directory ``~/ondemand/dev`` and the Develop dropdown will appear.
+
+.. warning:: If you do this, it is recommended that you treat the node that OnDemand is running on as a login node, as you are effectively giving those users shell access by letting them run arbitrary code on the OnDemand node (of course the UID of the processes are still their regular unprivileged user UID).
+
+If you do this, you still might want to restrict who sees the Develop dropdown in the Dashboard. To do that you can explicitly show or hide the dropdown in the Dashboard by setting ``Configuration.app_development_enabled`` to true based on one or more Ruby statements in the initializer ``/etc/ood/config/apps/dashboard/initializers/ood.rb``. Code in the initializer runs as the user. This code also has access to the `ood_support library `__ in which we provide some helper classes to work with User's and Groups. For example:
+
+
+ .. code-block:: ruby
+
+ Rails.application.config.after_initialize do
+ Configuration.app_development_enabled = OodSupport::Process.groups.include?(
+ OodSupport::Group.new("devgrp")
+ )
+ end
+
+ Or if you know the id of the group, this will avoid reading the ``/etc/group``
+ file:
+
+ .. code-block:: ruby
+
+ Rails.application.config.after_initialize do
+ Configuration.app_development_enabled = Process.groups.include?(5014)
+ end
+
+ Or a specific user list:
+
+ .. code-block:: ruby
+
+ Rails.application.config.after_initialize do
+ Configuration.app_development_enabled = %w(
+ bgohar efranz bmcmichael
+ ).include?(OodSupport::User.new.name)
+ end
diff --git a/downcase-ids/_sources/how-tos/app-development/interactive.rst.txt b/downcase-ids/_sources/how-tos/app-development/interactive.rst.txt
new file mode 100644
index 000000000..10f09e0e0
--- /dev/null
+++ b/downcase-ids/_sources/how-tos/app-development/interactive.rst.txt
@@ -0,0 +1,48 @@
+.. _app-development-interactive:
+
+Interactive Apps
+================
+
+Interactive apps can be developed and deployed using the same tools that are
+currently provided for all Open OnDemand applications but requires further
+:ref:`app-development-interactive-setup`.
+
+An Interactive App is a plugin that follows a custom
+file/directory structure and API that can be described by the five stages:
+:ref:`app-development-manifest`,
+:ref:`app-development-interactive-form`,
+:ref:`app-development-interactive-template`,
+:ref:`app-development-interactive-submit` and
+:ref:`app-development-interactive-view`.
+
+Additionally, there is :ref:`app-development-interactive-additional-info`.
+
+A typical file/directory structure for an Interactive App can look like::
+
+ my_app/
+ ├── form.yml
+ ├── manifest.yml
+ ├── submit.yml.erb
+ ├── template
+ │ ├── before.sh.erb
+ │ └── script.sh.erb
+ ├── view.html.erb
+ ├── info.{md,html}.erb
+ └── completed.{md,html}.erb
+
+Each of these files/directories are described below in their respective stage.
+
+.. toctree::
+ :maxdepth: 3
+ :caption: Stages of an Interactive App
+
+ interactive/manifest
+ interactive/form
+ interactive/form-widgets
+ interactive/dynamic-form-widgets
+ interactive/template
+ interactive/submit
+ interactive/view
+ interactive/sub-apps
+ interactive/conn-params
+ interactive/additional-info
diff --git a/downcase-ids/_sources/how-tos/app-development/interactive/additional-info.rst.txt b/downcase-ids/_sources/how-tos/app-development/interactive/additional-info.rst.txt
new file mode 100644
index 000000000..94a4d60fb
--- /dev/null
+++ b/downcase-ids/_sources/how-tos/app-development/interactive/additional-info.rst.txt
@@ -0,0 +1,47 @@
+.. _app-development-interactive-additional-info:
+
+Adding Additional Information to the session cards
+==================================================
+
+.. _bc_info_html_md_erb:
+
+info.{md,html}.erb
+------------------
+
+It's possible for you to add additional information to this session's card.
+
+You can do so by creating a Markdown file ``info.md.erb`` or an html file
+``info.html.erb`` in the applications folder. Markdown files get generated
+into html with # turning into an and ## turning into an and so on.
+
+Again, they're `eRuby (Embedded Ruby)`_ files so you can add some dynamic behavior
+to them. Along with any library you may choose to use you can also access these
+variables directly.
+
+id
+ The session UUID of the job
+cluster_id
+ The cluster the job was submitted to
+job_id
+ The job id from the scheduler
+created_at
+ The time the session was created
+
+
+.. _bc_completed_html_md_erb:
+
+completed.{md,html}.erb
+------------------------
+
+:ref:`bc_info_html_md_erb` above will display on the session's card
+regardless of the state of the job - it will always be displayed.
+
+``completed.{md,html}.erb`` on the other hand, will only display
+once the job has reached the ``completed`` state.
+
+You may want to add this to the session's card to display information
+to the user when the job is completed. Again, as it's `eRuby (Embedded Ruby)`_
+files so you can add some dynamic behavior to them.
+
+
+.. _eruby (embedded ruby): https://en.wikipedia.org/wiki/ERuby
diff --git a/downcase-ids/_sources/how-tos/app-development/interactive/conn-params.rst.txt b/downcase-ids/_sources/how-tos/app-development/interactive/conn-params.rst.txt
new file mode 100644
index 000000000..aa3e6f410
--- /dev/null
+++ b/downcase-ids/_sources/how-tos/app-development/interactive/conn-params.rst.txt
@@ -0,0 +1,70 @@
+.. _app-development-interactive-conn-params:
+
+Connection Parameters ``conn_params``
+=====================================
+
+App developers can use ``conn_params`` in the ``submit.yml.erb`` to pass runtime data generated
+in the ``before.sh.erb`` back to the ``view.html.erb``.
+
+This is helpful for:
+
+* Data that is only known **after** the job submits and starts running.
+* Data that needs to be used to connect to the application.
+
+This technique will generate a file in the jobs working directory called ``connection.yml``
+when the app launches which will contain the defined variables and their associated values.
+
+
+Jupyter Notebook Example
+------------------------
+
+Here's an example using a Jupyter application which needs
+needs to know the exact API to connect to. We can either connect to
+JuypterLab at ``/lab`` or Juypter Notebook at ``/tree``, but this
+information is not known until the job has been submitted.
+
+So once the job is submitted, we need to export the ``jupyter_api``
+environment variable that can then be written to ``connection.yml``
+which OnDemand will consume and use in the ``view.html.erb``.
+
+.. warning::
+
+ The environment variables in ``before.sh.erb`` *must* be lowercase and
+ exported through the *export* function.
+
+.. code:: shell
+
+ # within template/before.sh.erb
+
+ JUPYTER_API="<%= context.jupyterlab_switch == "1" ? "lab" : "tree" %>"
+
+ export jupyter_api="$JUPYTER_API"
+
+
+Now with that variable exported, you need to add it to ``conn_params`` in
+``submit.html.erb`` to ensure that OnDemand makes use of it.
+
+.. code::yaml
+
+ batch_connect:
+ template: "basic"
+ conn_params:
+ - jupyter_api
+
+In the ``view.html.erb``, which renders after the submission in the interactive apps page,
+you can access the value of this variable with::
+
+ <%-
+ ...
+ next_url = "#{base_url}/#{jupyter_api}"
+
+ full_url="#{login_url}?next=#{CGI.escape(next_url)}"
+ ...
+ %>
+
+ ...
+
+