Allow Fleet provision to run together with iotcored

docs/Fleet-provisioning.md

@@ -9,6 +9,21 @@ can get valid certificates. you can follow the link
 [here](https://docs.aws.amazon.com/greengrass/v2/developerguide/fleet-provisioning-setup.html)
 to learn how to create appropriate policies and claim certificate.
 
+```
+Note:
+Currently, fleet provisioning can only be run manually.
+Hence you will need to follow few important pre-steps
+
+1. Make sure you are logged in as root
+2. Allow read access to all user for your certificates
+    chmod -R +rx /ggcredentials/
+3. Make sure you do not fill iotCredEndpoint/iotDataEndpoint under
+  `aws.greengrass.NucleusLite` you should only fill these fields
+  under `aws.greengrass.fleet_provisioning`'s config
+4. If this is your not first run, remove the socket at
+    /run/greengrass/iotcoredfleet, if it exists
+```
+
 Sample Fleet provisioning template:
 
 ```json
@@ -73,52 +88,41 @@ config should roughly look as below.
 system:
   privateKeyPath: ""
   certificateFilePath: ""
-  rootCaPath: "/home/ubuntu/repo/fleetClaim/AmazonRootCA1.pem"
-  rootPath: "/home/ubuntu/aws-greengrass-lite/run_fleet/"
-  thingName: ""
+  rootCaPath: "/ggcredentials/fleetClaim/AmazonRootCA1.pem" #[Modify here]
+  rootPath: "/var/lib/greengrass/" #[Modify here]
+  thingName: "" #[Must leave blank]
 services:
   aws.greengrass.NucleusLite:
     componentType: "NUCLEUS"
     configuration:
       awsRegion: "us-east-1"
-      iotCredEndpoint: ""
-      iotDataEndpoint: ""
+      iotCredEndpoint: "" #[Must leave blank]
+      iotDataEndpoint: "" #[Must leave blank]
       iotRoleAlias: "GreengrassV2TokenExchangeRoleAlias"
       runWithDefault:
-        posixUser: "ubuntu:ubuntu"
+        posixUser: "user:group" #[Modify here]
       greengrassDataPlanePort: "8443"
-      tesCredUrl: "http://127.0.0.1:8080/"
   aws.greengrass.fleet_provisioning:
     configuration:
-      iotDataEndpoint: "dddddddddddddd-ats.iot.us-east-1.amazonaws.com"
-      iotCredEndpoint: "aaaaaaaaaaaaaa.credentials.iot.us-east-1.amazonaws.com"
-      claimKeyPath: "/home/ubuntu/fleetClaim/private.pem.key"
-      claimCertPath: "/home/ubuntu/fleetClaim/certificate.pem.crt"
-      templateName: "FleetTestNew"
-      templateParams: '{"SerialNumber": "14ALES55UFA"}'
-```
-
-With all this setup for IoT core now let's begin provisioning the device. First
-we will start an instance of ggconfigd
-
-```sh
-cd ./run
-../build/bin/ggconfigd
+      iotDataEndpoint: "aaaaaaaaaaaaaa-ats.iot.us-east-1.amazonaws.com" #[Modify here]
+      iotCredEndpoint: "cccccccccccccc.credentials.iot.us-east-1.amazonaws.com" #[Modify here]
+      claimKeyPath: "/ggcredentials/fleetClaim/private.pem.key" #[Modify here]
+      claimCertPath: "/ggcredentials/fleetClaim/certificate.pem.crt" #[Modify here]
+      templateName: "FleetTestNew" #[Modify here]
+      templateParams: '{"SerialNumber": "AAA55555"}' #[Modify here]
 ```
 
-In another shell, run the config script and the fleet provisioning
+In root user shell, run fleet provisioning
 
 ```sh
 cd ./run
-../build/bin/ggl-config-init --config ./init_config.yml
 ../build/bin/fleet-provisioning
 ```
 
 Now this will trigger the fleet provisioning script which will take a few
-minutes to complete, the shell doesn't automatically exits so look for a Info
-level log: `Process Complete, Your device is now provisioned`. then you can kill
-the process or wait for auto terminate of `300 seconds`.
+minutes to complete.
 
-You can then kill the config daemon as well.
+> Note: Device will reboot in case of successful run
 
-Now you can return to `## Running the nucleus` step in [SETUP.md](SETUP.md)
+If you are storing the standard output then look for log:
+`Process Complete, Your device is now provisioned`.

docs/design/executable/recipe-runner.md

@@ -1,27 +1 @@
 # `recipe-runner` design
-
-See [`recipe-runnerd` spec](../../spec/executable/recipe-runner.md) for the
-public interface for `recipe-runnerd`.
-
-`recipe-runner` is designed to run as a wrapper on the recipe's lifecycle script
-section. It will take the script section as is and will replace the recipe
-variables with appropriate values during runtime. The executable also
-understands the gg-global config and how to interact with it.
-
-Once a recipe is translated to a unit file, the selected lifecycle will be
-converted to a json file with its different lifecycle section. Each lifecycle
-section will generate a unit file that is suffixed with it's phase. For an
-example a recipe names `sampleComponent-0.1.0` will have unit files named
-`sampleComponent-0.1.0_install` and `sampleComponent-0.1.0_run` to represent
-install and run phase of lifecycle. As per the recipe2unit's design.
-
-Once a unit file is created `ggdeploymentd` will use `recipe2unit`'s functions
-to execute the specific unit file that will run provided lifecycle phase as a
-first time installation process.
-
-`recipe-runner` will use the provided selected lifecycle section and use
-`execvp` to execute the argument provided lifecycle section as a bash script. It
-will also forward any environment variables set during runtime. As a side effect
-it will create a temporary bash script file with all the gg-recipe variables
-replaced with appropriate actual values from the global config and then use will
-provide the newly created script file to `execvp`.

docs/examples/supported_lifecyle_types/1.json

@@ -0,0 +1,53 @@
+{
+  // Notice: All Key's are case sensitive
+  "RecipeFormatVersion": "2020-01-25",
+  "ComponentName": "s3.list.bucket.Python",
+  "ComponentVersion": "1.0.4",
+  "ComponentType": "aws.greengrass.generic",
+  "ComponentDescription": "This example Python component",
+  "ComponentPublisher": "TestUser",
+  "ComponentDependencies": {
+    "aws.greengrass.TokenExchangeService": {
+      "VersionRequirement": ">=0.0.0",
+      "DependencyType": "HARD"
+    },
+    "aws.greengrass.NucleusLite": {
+      "VersionRequirement": ">=2.0.0",
+      "DependencyType": "HARD"
+    }
+  },
+  "Manifests": [
+    {
+      "Platform": {
+        "os": "linux",
+        "runtime": "*", // Notice this is a required new field for GGLite
+        "architecture": "amd64"
+      },
+      "Lifecycle": {
+        "run": {
+          "RequiresPrivilege": true,
+          "Script": "python3 {artifacts:path}/python_list_S3_bucket.py"
+        },
+        "install": {
+          "RequiresPrivilege": true,
+          "Script": "pip install boto3"
+        }
+        // Notice bootstrap, shutdown and recover aren't currently supported
+      },
+      "Artifacts": [
+        {
+          // Notice URI needs to Uri(Camel case)
+          "Uri": "s3://gglite-artifact/python_list_S3_bucket.py",
+          "Digest": "---",
+          "Algorithm": "SHA-256",
+          "Unarchive": "NONE",
+          "Permission": {
+            "Read": "OWNER",
+            "Execute": "NONE"
+          }
+        }
+      ]
+    }
+  ],
+  "Lifecycle": {}
+}

docs/examples/supported_lifecyle_types/2.json

@@ -0,0 +1,153 @@
+{
+  // Notice: All Key's are case sensitive
+  "RecipeFormatVersion": "2020-01-25",
+  "ComponentName": "aws.greengrass.SecureTunneling",
+  "ComponentVersion": "1.0.100",
+  "ComponentType": "aws.greengrass.generic",
+  "ComponentDescription": "Enables AWS IoT Secure Tunneling connections that you can use to establish secure bidirectional communications with Greengrass core devices that are behind restricted firewalls.",
+  "ComponentPublisher": "AWS",
+  "ComponentConfiguration": {
+    "DefaultConfiguration": {
+      "accessControl": {
+        "aws.greengrass.ipc.mqttproxy": {
+          "aws.greengrass.SecureTunneling:mqttproxy:1": {
+            "policyDescription": "Access to tunnel notification pubsub topic",
+            "operations": ["aws.greengrass#SubscribeToIoTCore"],
+            "resources": ["$aws/things/+/tunnels/notify"]
+          }
+        }
+      },
+      "OS_DIST_INFO": "auto"
+    }
+  },
+  "ComponentDependencies": {
+    "aws.greengrass.NucleusLite": {
+      "VersionRequirement": ">=2.0.0",
+      "DependencyType": "SOFT"
+    }
+  },
+  "Manifests": [
+    {
+      "Platform": {
+        "os": "linux",
+        "runtime": "*",
+        "architecture": "amd64" // Notice regex isn't currently supported
+      },
+      "Lifecycle": {
+        "run": {
+          "Script": "java -jar {artifacts:path}/GreengrassV2SecureTunnelingComponent-1.0-all.jar linux x86_64"
+        }
+      },
+      "Artifacts": [
+        {
+          "Uri": "s3://gglite-artifact/GreengrassV2SecureTunnelingComponent-1.0-all.jar",
+          "Digest": "----",
+          "Algorithm": "SHA-256",
+          "Unarchive": "NONE",
+          "Permission": {
+            "Read": "OWNER",
+            "Execute": "NONE"
+          }
+        }
+      ]
+    },
+    {
+      "Platform": {
+        "os": "linux",
+        "runtime": "*",
+        "architecture": "x86_64" // Repeated Lifecycle with just architecture change
+      },
+      "Lifecycle": {
+        "run": {
+          "Script": "java -jar {artifacts:path}/GreengrassV2SecureTunnelingComponent-1.0-all.jar linux x86_64"
+        }
+      },
+      "Artifacts": [
+        {
+          "Uri": "s3://gglite-artifact/GreengrassV2SecureTunnelingComponent-1.0-all.jar",
+          "Digest": "----",
+          "Algorithm": "SHA-256",
+          "Unarchive": "NONE",
+          "Permission": {
+            "Read": "OWNER",
+            "Execute": "NONE"
+          }
+        }
+      ]
+    },
+    {
+      "Platform": {
+        "os": "linux",
+        "runtime": "*",
+        "architecture": "aarch64"
+      },
+      "Lifecycle": {
+        "run": {
+          "Script": "java -jar {artifacts:path}/GreengrassV2SecureTunnelingComponent-1.0-all.jar linux aarch64"
+        }
+      },
+      "Artifacts": [
+        {
+          "Uri": "s3://gglite-artifact/GreengrassV2SecureTunnelingComponent-1.0-all.jar",
+          "Digest": "----",
+          "Algorithm": "SHA-256",
+          "Unarchive": "NONE",
+          "Permission": {
+            "Read": "OWNER",
+            "Execute": "NONE"
+          }
+        }
+      ]
+    },
+    {
+      "Platform": {
+        "os": "linux",
+        "runtime": "*",
+        "architecture": "armv8"
+      },
+      "Lifecycle": {
+        "run": {
+          "Script": "java -jar {artifacts:path}/GreengrassV2SecureTunnelingComponent-1.0-all.jar linux aarch64"
+        }
+      },
+      "Artifacts": [
+        {
+          "Uri": "s3://gglite-artifact/GreengrassV2SecureTunnelingComponent-1.0-all.jar",
+          "Digest": "----",
+          "Algorithm": "SHA-256",
+          "Unarchive": "NONE",
+          "Permission": {
+            "Read": "OWNER",
+            "Execute": "NONE"
+          }
+        }
+      ]
+    },
+    {
+      "Platform": {
+        "os": "linux",
+        "architecture.detail": "armv7l",
+        "runtime": "*",
+        "architecture": "arm"
+      },
+      "Lifecycle": {
+        "run": {
+          "Script": "java -jar {artifacts:path}/GreengrassV2SecureTunnelingComponent-1.0-all.jar linux armv7l"
+        }
+      },
+      "Artifacts": [
+        {
+          "Uri": "s3://gglite-artifact/GreengrassV2SecureTunnelingComponent-1.0-all.jar",
+          "Digest": "----",
+          "Algorithm": "SHA-256",
+          "Unarchive": "NONE",
+          "Permission": {
+            "Read": "OWNER",
+            "Execute": "NONE"
+          }
+        }
+      ]
+    }
+  ],
+  "Lifecycle": {}
+}

docs/examples/supported_lifecyle_types/3.yaml

@@ -0,0 +1,43 @@
+---
+# Notice: All Key's are case sensitive
+RecipeFormatVersion: "2020-01-25"
+ComponentName: "s3.list.bucket.Python"
+ComponentVersion: "1.0.4"
+ComponentType: "aws.greengrass.generic"
+ComponentDescription: "This example Python component"
+ComponentPublisher: "Amazon"
+ComponentDependencies:
+  aws.greengrass.TokenExchangeService:
+    VersionRequirement: ">=0.0.0"
+    DependencyType: "HARD"
+  aws.greengrass.NucleusLite:
+    VersionRequirement: ">=2.0.0"
+    DependencyType: "HARD"
+ComponentConfiguration:
+  DefaultConfiguration:
+    containerMode: "auto"
+Manifests:
+  - Platform:
+      os: "linux"
+      runtime: "aws_nucleus_lite" # Notice this is a required new field for GGLite
+    Lifecycle: {} #Notice No Lifecycle or Selection so Global lifecycle is used
+    Artifacts:
+      - Uri: "s3://gglite-artifact/python_list_S3_bucket.py"
+        Digest: "----"
+        Algorithm: "SHA-256"
+        Unarchive: "NONE"
+        Permission:
+          Read: "OWNER"
+          Execute: "NONE"
+Lifecycle:
+  # Notice all is case sensitive and is required when no selection is provided
+  - all:
+      Setenv:
+        TEST_VALUE: "{configuration:/containerMode}"
+      run:
+        # Notice Skipif is not supported
+        RequiresPrivilege: true
+        Script: "python3 {artifacts:path}/python_list_S3_bucket.py"
+        Timeout: 200 #Seconds
+        Setenv:
+          TEST_VALUE: sample