updated ReadMe

README.md

@@ -1,8 +1,8 @@
-## HPE Docker Volume Plugin for HPE 3PAR StoreServ
+## HPE 3PAR Volume Plugin for Docker
 
 HPE Docker Volume Plugin is an open source project that provides persistent storage and features for your containerized applications using HPE 3PAR StoreServ Storage arrays.
 
-The HPE Docker Volume Plugin supports popular container platforms like Docker, Kubernetes, OpenShift and SuSE CaaS/CAP 
+The HPE Docker Volume Plugin supports popular container platforms like Docker, Kubernetes, OpenShift 
 
 ## HPE Docker Volume Plugin Overview
 
@@ -40,6 +40,10 @@ Here is an example of the HPE Docker Volume plugin being used in an OpenShift en
   * snapshot mount
   * mount_conflict_delay
   * concurrent volume access
+  * replication
+  * snapshot schedule
+  * file system permissions and ownership
+  * multiple backends
 
 ## Usage
 
@@ -48,3 +52,53 @@ See the [usage guide](/docs/usage.md) for details on the supported operations an
 ## Troubleshooting
 
 Troubleshooting issues with the plugin can be performed using these [tips](/docs/troubleshooting.md)
+
+
+## SPOCK Link for HPE 3PAR Volume Plugin for Docker
+
+* [SPOCK Link](https://spock.corp.int.hpe.com/spock/utility/document.aspx?docurl=Shared%20Documents/hw/3par/3par_volume_plugin_for_docker.pdf)
+
+## Limitations
+- List of issues around the containerized version of the plugin/Managed plugin is present in https://github.com/hpe-storage/python-hpedockerplugin/issues 
+
+- ``$ docker volume prune`` is not supported for volume plugin, instead use ``$docker volume rm $(docker volume ls -q -f "dangling=true") `` to clean up orphaned volumes.
+
+- Shared volume support is present for containers running on the same host.
+
+- For upgrading the plugin from older version to the current released version, user needs to unmount all the volumes and follow the standard
+ upgrade procedure described in docker guide. 
+
+- Volumes created using older plugins (2.0.2 or below) do not have snap_cpg associated with them, hence when the plugin is upgraded to      2.1 and user wants to perform clone/snapshot operations on these old volumes, he/she must set the snap_cpg for the
+   corresponding volumes using 3par cli or any tool before performing clone/snapshot operations.
+
+- While inspecting a snapshot, its provisioning field is set to that of parent volume's provisioning type. In 3PAR however, it is shown as 'snp'.
+
+- Mounting a QoS enabled volume can take longer than a volume without QoS for both FC and iSCSI protocol.
+
+- For a cloned volume with the same size as source volume, comment field won’t be populated on 3PAR.
+
+- User not allowed to import a 3PAR legacy volume when it is under use(Active VLUN).
+
+- User needs to explicitly manage all the child snapshots, until which managed parent volume cannot be deleted
+
+- User cannot manage already managed volume by other docker host(i.e. volume thats start with 'dcv-')
+
+- It is recommended for a user to avoid importing legacy volume which has schedules associated with it. If this volume needs to be imported please remove existing schedule on 3PAR and import the legacy volume.
+
+- "Snapshot schedule creation can take more time resulting into Docker CLI timeout. However, snapshot schedule may still get created in the background. User can follow below two steps in case of time out issue from docker daemon while creating snapshot schedule."
+
+```Inspect the snapshot to verify if the snapshot schedule got created
+docker volume inspect <snapshot_name>. This should display snapshot details with snapshot schedule information.
+
+Verify if schedule got created on the array using 3PAR CLI command:
+$ showsched
+```
+
+- If a mount fails due to dangling LUN use this section of troubleshooting guide [Removing Dangling LUN](https://github.com/hpe-storage/python-hpedockerplugin/blob/master/docs/troubleshooting.md#removing-dangling-lun)
+
+- If two or more backends are defined with the same name then the last backend is picked up and rest ignored.
+
+- after doing scsi rescan if the symlinks for the device are not populated in /dev/disk/by-path, Plugin will not function correctly during mount operation.
+
+- For volume uppersize limiation, please do refer 3PAR's documentation.
+

ansible_3par_docker_plugin/README.md

@@ -1,79 +1,164 @@
 # Automated Installer for 3PAR Docker Volume plugin (Ansible)
 
-These are Ansible playbooks to automate the install of the HPE 3PAR Docker Volume Plug-in for Docker for use within Kubernetes/OpenShift environments.
+These are Ansible playbooks to automate the install of the HPE 3PAR Docker Volume Plug-in for Docker for use within standalone docker environment or Kubernetes/OpenShift environments.
 
-If you are not using Kubernetes or OpenShift, we recommend you take a look at the [Quick Start guide](/docs/quick_start_guide.md) for using the HPE 3PAR Docker Volume Plug-in in a standalone Docker environment.
-
->**NOTE:** The Ansible installer only supports RHEL/CentOS. If you are using another distribution of Linux, you will need to modify the playbooks to support your application manager (apt, etc.) and the pre-requisite packages.
+>**NOTE:** The Ansible installer only supports Ubuntu/RHEL/CentOS. If you are using another distribution of Linux, you will need to modify the playbooks to support your application manager (apt, etc.) and the pre-requisite packages.
 
 ### Getting Started
 
 These playbooks perform the following tasks on the Master/Worker nodes as defined in the Ansible [hosts](/ansible_3par_docker_plugin/hosts) file.
 * Configure the Docker Services for the HPE 3PAR Docker Volume Plug-in
-* Deploys a 3-node Highly Available etcd cluster
 * Deploys the config files (iSCSI or FC) to support your environment
-* Installs the HPE 3PAR Docker Volume Plug-in (Containerized version) for Kubernetes/OpenShift
-* Deploys the HPE FlexVolume Drivers
+* Installs the HPE 3PAR Docker Volume Plug-in (Containerized version)
+* For standalone docker environment,
+  * Deploys an etcd cluster
+* For Kubernetes/OpenShift, 
+  * Deploys a Highly Available etcd cluster used by the HPE 3PAR Docker Volume plugin 
+    * Supports single node (Use only for testing purposes) or multi-node deployment (HA) as defined in the Ansible hosts file
+  * Deploys the HPE FlexVolume Driver
 
 ### Prerequisites:
-
-  - Install Ansible per [Installation Guide](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html)
-  - Login to 3PAR to create known_hosts file
-  > **Note:** Entries for the Master and Worker nodes should already exist within the /<user>/.ssh/known_hosts file from the OpenShift installation. If not, you will need to log into each of the Master and Worker nodes as well to prevent connection errors from Ansible.
-
-  - modify files/hpe.conf ([iSCSI](/ansible_3par_docker_plugin/files/iSCSI_hpe.conf) or [FC](/ansible_3par_docker_plugin/files/FC_hpe.conf)) based on your HPE 3PAR Storage array configuration. An example can be found here: [sample_hpe.conf](/ansible_3par_docker_plugin/files/sample_hpe.conf)
+  - Install Ansible 2.5 or above as per [Installation Guide](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html)
+
+  - Login to 3PAR via SSH to create entry in /\<user>\/.ssh/known_hosts file
+  > **Note:** Entries for the Master and Worker nodes should already exist within the /\<user>\/.ssh/known_hosts file from the OpenShift installation. If not, you will need to log into each of the Master and Worker nodes as well to prevent connection errors from Ansible.
+
+  - Clone the python-hpedockerplugin repository
+    ```
+    git clone https://github.com/hpe-storage/python-hpedockerplugin
+    cd python-hpedockerplugin/ansible_3par_docker_plugin
+    ```
+
+  - Add [plugin configuration properties - sample](/ansible_3par_docker_plugin/properties/plugin_configuration_properties_sample.yml) at `properties/plugin_configuration_properties.yml` based on your HPE 3PAR Storage array configuration. Some of the properties are mandatory and must be specified in the properties file while others are optional. 
+
+      | Property  | Mandatory | Default Value | Description |
+      | ------------- | ------------- | ------------- | ------------- |
+      | ```hpedockerplugin_driver```  | Yes  | No default value  | ISCSI/FC driver  (hpedockerplugin.hpe.hpe_3par_iscsi.HPE3PARISCSIDriver/hpedockerplugin.hpe.hpe_3par_fc.HPE3PARFCDriver) |
+      | ```hpe3par_ip```  | Yes  | No default value | IP address of 3PAR array |
+      | ```hpe3par_username```  | Yes  | No default value | 3PAR username |
+      | ```hpe3par_password```  | Yes  | No default value | 3PAR password |
+      | ```hpe3par_cpg```  | Yes  | No default value | Primary user CPG |
+      | ```volume_plugin```  | Yes  | No default value | Name of the docker volume image (only required with DEFAULT backend) |
+      | ```encryptor_key```  | No  | No default value | Encryption key string for 3PAR password |
+      | ```logging```  | No  | ```INFO``` | Log level |
+      | ```hpe3par_debug```  | No  | No default value | 3PAR log level |
+      | ```suppress_requests_ssl_warning```  | No  | ```True``` | Suppress request SSL warnings |
+      | ```hpe3par_snapcpg```  | No  | ```hpe3par_cpg``` | Snapshot CPG |
+      | ```hpe3par_iscsi_chap_enabled```  | No  | ```False``` | ISCSI chap toggle |
+      | ```hpe3par_iscsi_ips```  | No  |No default value | Comma separated iscsi port IPs (only required if driver is ISCSI based) |
+      | ```use_multipath```  | No  | ```False``` | Mutltipath toggle |
+      | ```enforce_multipath```  | No  | ```False``` | Forcefully enforce multipath |
+      | ```ssh_hosts_key_file```  | No  | ```/root/.ssh/known_hosts``` | Path to hosts key file |
+      | ```quorum_witness_ip```  | No  | No default value | Quorum witness IP |
+      | ```mount_prefix```  | No  | No default value | Alternate mount path prefix |
+      | ```hpe3par_iscsi_ips```  | No  | No default value | Comma separated iscsi IPs. If not provided, all iscsi IPs will be read from the array and populated in hpe.conf |
+      | ```vlan_tag```  | No  | False | Populates the iscsi_ips which are vlan tagged, only applicable if ```hpe3par_iscsi_ips``` is not specified |
+      | ```replication_device```  | No  | No default value | Replication backend properties |
+
+  - The Etcd ports can be modified in [etcd cluster properties](/ansible_3par_docker_plugin/properties/etcd_cluster_properties.yml) as follows:
+
+      | Property  | Mandatory | Default Value |
+      | ------------- | ------------- | ------------- |
+      | ```etcd_peer_port```  | Yes  | 23800  |
+      | ```etcd_client_port_1```  | Yes  | 23790 |
+      | ```etcd_client_port_2```  | Yes  | 40010 |
+
+    > **Note:** Please ensure that the ports specified above are unoccupied before installation. If the ports are not available on a particular node, etcd installation will fail.
+
+    > **Limitation:** The installer, in the current state does not have the capability to add or remove nodes in the etcd cluster. In case an etcd node is not responding or goes down, it is beyond the current scope to admit it back into the cluster. Please follow the [etcd documentation](https://coreos.com/etcd/docs/latest/etcd-live-cluster-reconfiguration.html) to do so manually.
+
+  - It is recommended that the properties file is [encrypted using Ansible Vault](/ansible_3par_docker_plugin/encrypt_properties.md).
 
   - Modify [hosts](/ansible_3par_docker_plugin/hosts) file to define your Master/Worker nodes as well as where you want to deploy your etcd cluster
+
+### Working with proxies:
+
+Set `http_proxy` and `https_proxy` in the [inventory hosts file](/ansible_3par_docker_plugin/hosts) while installing plugin on Kubernetes/Openshift setup. For setting proxies in the standalone plugin installation, see [inventory hosts file for standalone plugin installation](/ansible_3par_docker_plugin/hosts_standalone_nodes)
 
 ### Usage
 
 Once the prerequisites are complete, run the following command:
 
+- Fresh installation on standalone docker environment:
 ```
-$ ansible-playbook -i hosts install_hpe_3par_volume_driver.yml
-```
-
-Once complete you will be ready to start using the HPE 3PAR Docker Volume Plug-in within Kubernetes/OpenShift.
-
-Please refer to the Kubernetes/OpenShift section in the [Usage Guide](/docs/usage.md#k8_usage) on how to create and deploy some sample SCs, PVCs, and Pods with persistent volumes using the HPE 3PAR Docker Volume Plug-in.
-
-
-<br><br>
-
-
-### Known Issues
-
-Ansible on some Linux Distros (i.e. CentOS and Ubuntu) may throw an error about missing the `docker` module.
-
-```
-TASK [run etcd container] ******************************************************************************************************************************************
-fatal: [192.168.1.35]: FAILED! => {"changed": false, "msg": "Failed to import docker-py - No module named docker. Try `pip install docker-py`"}
-```
-
-Run:
-
-```
-pip install docker
+$ cd python-hpedockerplugin/ansible_3par_docker_plugin
+$ ansible-playbook -i hosts_standalone_nodes install_standalone_hpe_3par_volume_driver.yml --ask-vault-pass
 ```
 
------------------------------------------------------------------------------------
-
-On Ansible 2.6 and later, per https://github.com/ansible/ansible/issues/42162, `docker-py` has been deprecated and when running the Ansible playbook, you may see the following error:
-
+- Fresh installation on Openshift/Kubernetes environment:
 ```
-docker_container: create_host_config() got an unexpected keyword argument 'init'
+$ cd python-hpedockerplugin/ansible_3par_docker_plugin
+$ ansible-playbook -i hosts install_hpe_3par_volume_driver.yml --ask-vault-pass
 ```
+> **Note:** ```--ask-vault-pass``` is required only when the properties file is encrypted
+
+
+Once complete you will be ready to start using the HPE 3PAR Docker Volume Plug-in.
+
+- Update the array backends in Standalone/Openshift/Kubernetes environment:
+  * Modify the [plugin configuration properties - sample](/ansible_3par_docker_plugin/properties/plugin_configuration_properties_sample.yml) at `properties/plugin_configuration_properties.yml` based on the updated HPE 3PAR Storage array configuration. Additional backends may be added or removed from the existing configuration. Individual attributes of the existing array configuration may also be modified.
+
+    * Update array backend on standalone docker environment:
+    ```
+    $ cd python-hpedockerplugin/ansible_3par_docker_plugin
+    $ ansible-playbook -i hosts_standalone_nodes install_standalone_hpe_3par_volume_driver.yml --ask-vault-pass
+    ```
+
+    * Update array backend on Openshift/Kubernetes environment:
+    ```
+    $ cd python-hpedockerplugin/ansible_3par_docker_plugin
+    $ ansible-playbook -i hosts install_hpe_3par_volume_driver.yml --ask-vault-pass
+    ```
+  > **Note:** It is not recommended to change the Etcd information and array encryption password during the backend update process
+
+- Upgrade the docker volume plugin
+  * Modify the `volume_plugin` in [plugin configuration properties - sample](/ansible_3par_docker_plugin/properties/plugin_configuration_properties_sample.yml) and point it to the latest image from docker hub
+      * Update plugin on standalone docker environment:
+      ```
+      $ cd python-hpedockerplugin/ansible_3par_docker_plugin
+      $ ansible-playbook -i hosts_standalone_nodes install_standalone_hpe_3par_volume_driver.yml --ask-vault-pass
+      ```
+     * Update plugin on Openshift/Kubernetes environment:
+     ```
+     $ cd python-hpedockerplugin/ansible_3par_docker_plugin
+     $ ansible-playbook -i hosts install_hpe_3par_volume_driver.yml --ask-vault-pass
+     ```
+   > **Note:** 
+     - Ensure that all the nodes in the cluster are present in the inventory [hosts](/ansible_3par_docker_plugin/hosts) file
+     - The docker volume plugin will be restarted and the user will not be able to create the volume during the process
+
+   * Successful upgrade will remove the old plugin container and replace it with the new plugin container which is specified in the plugin properties file 
+
+- Install docker volume plugin to additional nodes in the cluster
+  * Add the new nodes in the respective sections in the inventory [hosts](/ansible_3par_docker_plugin/hosts) file
+  * Only new nodes IP or hostnames must be present in the hosts file
+  * Do not change the etcd hosts from the existing setup. Do not add or remove nodes in the etcd section
+
+     * Install plugin on new nodes on Openshift/Kubernetes environment:
+     ```
+     $ cd python-hpedockerplugin/ansible_3par_docker_plugin
+     $ ansible-playbook -i hosts install_hpe_3par_volume_driver.yml --ask-vault-pass
+     ```
+
+     * Uninstall plugin on nodes on Openshift/Kubernetes environment:
+     ```
+     $ cd python-hpedockerplugin/ansible_3par_docker_plugin
+     $ ansible-playbook -i hosts uninstall/uninstall_hpe_3par_volume_driver.yml --ask-vault-pass
+     ```
+
+     * Uninstall plugin along with etcd on nodes on Openshift/Kubernetes environment:
+     ```
+     $ cd python-hpedockerplugin/ansible_3par_docker_plugin
+     $ ansible-playbook -i hosts uninstall/uninstall_hpe_3par_volume_driver_etcd.yml --ask-vault-pass
+     ```
+
+     > **Note:** This process only adds or removes docker volume plugin and/or etcd in nodes in an existing cluster. It does not add or remove nodes in Kubernetes/Openshift cluster
+   * On success after adding plugin on new nodes, the additional nodes will have a running docker volume plugin container
+   * On success after removing plugin from specified nodes, docker volume plugin container will be removed
+
+Please refer to [Usage Guide](/docs/usage.md) on how to perform volume related actions on the standalone docker environment.
 
-`docker-py` is no longer supported and has been deprecated in favor of the `docker` module.
-
-If `docker-py` is installed, run:
-
-```
-pip uninstall docker-py
-```
+Please refer to the Kubernetes/OpenShift section in the [Usage Guide](/docs/usage.md#k8_usage) on how to create and deploy some sample SCs, PVCs, and Pods with persistent volumes using the HPE 3PAR Docker Volume Plug-in.
 
-Run:
 
-```
-pip install docker
-```
+<br><br>

ansible_3par_docker_plugin/container/Dockerfile

@@ -0,0 +1,29 @@
+FROM alpine:3.8
+
+MAINTAINER Farhan Nomani <[email protected]>
+
+RUN echo "===> Installing sudo to emulate normal OS behavior..."  && \
+    apk --update add sudo                                         && \
+    \
+    \
+    echo "===> Adding Python runtime..."  && \
+    apk --no-cache add ca-certificates && \
+    apk --update add python py-pip openssl unzip && \
+    apk --update add --virtual build-dependencies wget \
+                openssh-keygen openssh-server openssh-client \
+                python-dev libffi-dev openssl-dev build-base  && \
+    pip install --upgrade pip pycrypto cffi                   && \
+    \
+    \
+    echo "===> Installing Ansible..."  && \
+    pip install ansible         && \
+    \
+    wget https://github.com/hpe-storage/python-hpedockerplugin/archive/master.zip
+
+RUN unset http_proxy
+RUN unset https_proxy
+RUN unzip master.zip
+
+RUN rm -f master.zip
+RUN mv python-hpedockerplugin-master/ansible_3par_docker_plugin/ .
+RUN rm -rf python-hpedockerplugin-master/

ansible_3par_docker_plugin/container/README.md

@@ -0,0 +1,14 @@
+# Docker Volume Plugin Installer - Docker Image
+
+This is an Alpine based image with Ansible and its dependency installed along with the latest HPE 3PAR docker volume plugin ansible installer tasks and playbooks.
+
+Usage:
+
+- Run the latest docker image from docker hub, the command below will run the pre built container and open shell
+  - `docker run -it hpestorage/legacyvolumeplugininstaller /bin/sh`
+
+- Set the proxy (if required)
+  - `export http_proxy=<your-proxy>:<your-port>`
+  - `export https_proxy=<your-proxy>:<your-port>`
+
+- Follow [this link](/ansible_3par_docker_plugin/README.md) to set the node information, backend properties and run the installation playbook