diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md index 9ff565ddc..fbba6fede 100644 --- a/RELEASE_NOTES.md +++ b/RELEASE_NOTES.md @@ -11,13 +11,13 @@ Detailed information can be found [here](./docs/RECIPE_SUPPORT_CHANGES.md). ## Installing from source -To install Greengrass nucleus lite from source, please follow the setup guide -[SETUP.md](./docs/SETUP.md) and [TES.md](./docs/TES.md). Once the development -environment is setup, please refer to [INSTALL.md](./docs/INSTALL.md). +To install Greengrass nucleus lite from source, please follow the installation +guide [INSTALL.md](./docs/INSTALL.md) and [TES.md](./docs/TES.md). Once the +development environment is setup, please refer to [SETUP.md](./docs/SETUP.md). For provisioning Greengrass nucleus lite devices by claim certificates, please take a look at the fleet provisioning by claim setup guide -[here](./docs/Fleet-provisioning.md). +[here](./docs/FLEET_PROVISIONING.md). ## Contribution guidelines diff --git a/docs/Fleet-provisioning.md b/docs/FLEET_PROVISIONING.md similarity index 57% rename from docs/Fleet-provisioning.md rename to docs/FLEET_PROVISIONING.md index 1dea500a4..1311b818a 100644 --- a/docs/Fleet-provisioning.md +++ b/docs/FLEET_PROVISIONING.md @@ -9,23 +9,27 @@ can get valid certificates. you can follow the link [here](https://docs.aws.amazon.com/greengrass/v2/developerguide/fleet-provisioning-setup.html) to learn how to create appropriate policies and claim certificate. -``` -Note: -Currently, fleet provisioning can only be run manually. -Hence you will need to follow few important pre-steps - -1. Make sure you are logged in as root -2. Allow read access to all user for your certificates - chmod -R +rx /ggcredentials/ -3. Make sure you do not fill iotCredEndpoint/iotDataEndpoint under - `aws.greengrass.NucleusLite` you should only fill these fields - under `aws.greengrass.fleet_provisioning`'s config +Greengrass nucleus lite generates csr and private keys locally and then sends +the csr to iotcore to generate a certificate. This behavior is different from +Greengrass classic. Hence, make sure your claim certificate has connect, +publish, subscribe and receive access to `CreateCertificateFromCsr` and +`RegisterThing` topics mentioned in +[linked AWS docs](https://docs.aws.amazon.com/iot/latest/developerguide/fleet-provision-api.html). + +## Before getting started: + +Currently, fleet provisioning can only be run manually. Hence you will need to +follow few important pre-steps + +1. This section assumes that the system has already met the dependencies + mentioned in [SETUP.md](./SETUP.md#dependencies). +2. Make sure you are logged in as root. +3. Make sure you do not fill `iotCredEndpoint/iotDataEndpoint` under + `aws.greengrass.NucleusLite` you should only fill these fields under + `aws.greengrass.fleet_provisioning`'s config. See the + [sample config below](#configyaml). 4. If this is your not first run, remove the socket at - /run/greengrass/iotcoredfleet, if it exists -5. Fleet provisioning assumes the your GGL_SYSTEMD_SYSTEM_USER - and GGL_SYSTEMD_SYSTEM_GROUP to be ggcore:ggcore please change - appropriately if you change these values during compile time -``` + `/run/greengrass/iotcoredfleet`, if it exists. Sample Fleet provisioning template: @@ -82,15 +86,19 @@ Sample Fleet provisioning template: } ``` +## Setting up the device side for provisioning + Here we can assume your template name is `FleetTestNew` and your template -requires you to only provide a serial number as parameter. Then your nucleus -config should roughly look as below. +requires(based on above template) you to only provide a serial number as +parameter. Then your nucleus config should roughly look as below: + +### `config.yaml` ```yaml --- system: - privateKeyPath: "" - certificateFilePath: "" + privateKeyPath: "" #[Must leave blank] + certificateFilePath: "" #[Must leave blank] rootCaPath: "/ggcredentials/fleetClaim/AmazonRootCA1.pem" #[Modify here] rootPath: "/var/lib/greengrass/" #[Modify here] thingName: "" #[Must leave blank] @@ -101,7 +109,7 @@ services: awsRegion: "us-east-1" iotCredEndpoint: "" #[Must leave blank] iotDataEndpoint: "" #[Must leave blank] - iotRoleAlias: "GreengrassV2TokenExchangeRoleAlias" + iotRoleAlias: "GreengrassV2TokenExchangeRoleAlias" #[Modify if needed] runWithDefault: posixUser: "user:group" #[Modify here] greengrassDataPlanePort: "8443" @@ -115,17 +123,33 @@ services: templateParams: '{"SerialNumber": "AAA55555"}' #[Modify here] ``` -In root user shell, run fleet provisioning +Once completed, the config needs to be moved and all the services need to be +started (if not started already). Run the following command for it, assuming +your current working directory is root of greengrass repository: ```sh -cd ./run -../build/bin/fleet-provisioning +$ mkdir -p /etc/greengrass +$ cp ./run/config.yaml /etc/greengrass/config.yaml +$ ./misc/run_nucleus +``` + +In root user shell, run the fleet provisioning binary. + +If you changed `GGL_SYSTEMD_SYSTEM_USER` and `GGL_SYSTEMD_SYSTEM_GROUP` +mentioned in [CMakeLists.txt](../CMakeLists.txt), you can override default by +adding `-u "ggcore:ggcore"` at the end of following command: + +```sh +$ ../build/bin/fleet-provisioning ``` Now this will trigger the fleet provisioning script which will take a few minutes to complete. -> Note: Device will reboot in case of successful run +> Note: Device will reboot in case of a successful run. If you are storing the standard output then look for log: `Process Complete, Your device is now provisioned`. + +> You might see some error log such as `process is getting kill by signal 15` +> this is expected and correct behavior. diff --git a/docs/INSTALL.md b/docs/INSTALL.md index 93f45a814..0483efeb6 100644 --- a/docs/INSTALL.md +++ b/docs/INSTALL.md @@ -15,6 +15,9 @@ sudo apt install libssl-dev libcurl4-openssl-dev libsqlite3-dev libyaml-dev \ libsystemd-dev liburiparser-dev uuid-dev libevent-dev libzip-dev cgroup-tools ``` +The versions required by greengrass nucleus lite are mentioned at +[SETUP.md](./SETUP.md#dependencies). + ## Build tools To build the project, you will need the following build dependencies: diff --git a/docs/SETUP.md b/docs/SETUP.md index 281694720..b4d0e463f 100644 --- a/docs/SETUP.md +++ b/docs/SETUP.md @@ -33,29 +33,35 @@ This project uses the following third party library dependencies: ## Configuring Greengrass You may configure a single device with the instruction below or a fleet of -devices with the steps from [Fleet Provisioning guide](Fleet-provisioning.md). +devices with the steps from [Fleet Provisioning guide](FLEET_PROVISIONING.md). Choose one or the other. To configure Greengrass, you will need a config YAML file, in the same format as the Classic nucleus config. An example config file is available in -[`doc/examples/sample_nucleus_config.yml`](examples/sample_nucleus_config.yml). +[`docs/examples/sample_nucleus_config.yaml`](examples/sample_nucleus_config.yaml). If this is the first time you are creating a GG device, please follow the instruction in the [TES setup instructions](./TES.md) to get a role alias, thing, certificate, private key, and endpoints for your device. -Make a copy of the [sample configuration](./examples/sample_nucleus_config.yml). +Make a copy of the [sample configuration](./examples/sample_nucleus_config.yaml) +as `config.yaml`. + +```sh +cp docs/examples/sample_nucleus_config.yaml ./config.yaml +``` Configure the following in your config file - privateKeyPath: Path to private key for the Thing - certificateFilePath: Path to Thing certificate -- thingName: Name of the Thing +- rootCaPath: Path to Amazon Root CA certificate - rootPath: Absolute path to the Greengrass rootpath directory +- thingName: Name of the Thing - awsRegion: The AWS region with the Thing - iotCredEndpoint: The IoT Core endpoint - iotDataEndpoint: The IoT Core endpoint -- posixUser: Colon separated user/group that generic components should run as - iotRoleAlias: The name of the role alias for accessing TES +- posixUser: Colon separated user/group that generic components should run as `posixUser` must be set to a valid user and group. If no colon and group is provided, the user's default group is used. If not running Greengrass as root, @@ -67,11 +73,13 @@ as `/etc/greengrass/config.yaml`, and/or in one or more files in The config daemon will initially load `/etc/greengrass/config.yaml` and then update the initial configuration with any other config files present in -`/etc/greengrass/config.d/` +`/etc/greengrass/config.d/`. Copy your configuration file to the above directory +( Note: you might need to run with `sudo` in case you are getting +`permission denied` error) - ```sh mkdir -p /etc/greengrass -cp ./init_config.yml /etc/greengrass/config.yaml +cp ./config.yaml /etc/greengrass/config.yaml ``` ## Running the nucleus @@ -120,8 +128,3 @@ With the above, you can start a local deployment with: --artifacts-dir ~/sample-component/artifacts \ --add-component com.example.SampleComponent=1.0.0 ``` - -## Local-Deploying the sample Hello World component - -See the -[com.example.LiteHelloWorld component README](../hello-world-component/README.md) diff --git a/docs/examples/sample.ggLitePython/README.md b/docs/examples/sample.ggLitePython/README.md index bef33edcf..d26e3c4a9 100644 --- a/docs/examples/sample.ggLitePython/README.md +++ b/docs/examples/sample.ggLitePython/README.md @@ -57,9 +57,9 @@ You may deploy the componet the following ways: - Uncomment the `Artifacts` section of recipe. - Provide S3 url under `Uri` section in recipe. -## End result +## After deploying the component -The logs for following the deployment status can be found by: +The logs for to follow the deployment progress can be accessed by: ```shell $ journalctl -xeau ggl.core.ggdeploymentd.service @@ -72,5 +72,5 @@ S3 bucket names that exist in your aws account. $ journalctl -xeau ggl.sample.ggLitePython.service ``` -> If you do see the list of names then keep on press up arrow key until you see -> `HELLO WORLD` text in your logs +> If you do not see the list of names then continue pressing the up arrow key +> until you see `HELLO WORLD` text in your logs diff --git a/docs/examples/sample_nucleus_config.yml b/docs/examples/sample_nucleus_config.yaml similarity index 100% rename from docs/examples/sample_nucleus_config.yml rename to docs/examples/sample_nucleus_config.yaml diff --git a/docs/spec/executable/fleet-provisioning.md b/docs/spec/executable/fleet-provisioning.md index c7d38c845..ccd10eda4 100644 --- a/docs/spec/executable/fleet-provisioning.md +++ b/docs/spec/executable/fleet-provisioning.md @@ -28,7 +28,7 @@ operations - [fleet-provisioning-param-certfilePath-1] The argument will provide the path to the certificate that will be created locally as well as the onces that will - be fetched from iot core. By deafult the location will be the current working + be fetched from iot core. By default the location will be the current working directory - [fleet-provisioning-param-certfilePath-2] The certfilePath argument can be provided by `--cert-file-Path` or `-c`. diff --git a/fleet-provisioning/bin/fleet-provisioning.c b/fleet-provisioning/bin/fleet-provisioning.c index e044bb7c7..c560411b0 100644 --- a/fleet-provisioning/bin/fleet-provisioning.c +++ b/fleet-provisioning/bin/fleet-provisioning.c @@ -18,35 +18,50 @@ static char doc[] = "fleet provisioner -- Executable to automatically " "provision the device to AWS IOT core"; static const char COMPONENT_NAME[] = "fleet-provisioning"; -static struct argp_option opts[] = { - { "claim-key", - 'k', - "path", - 0, - "Path to key for client claim private certificate", - 0 }, - { "claim-cert", - 'c', - "path", - 0, - "Path to key for client claim certificate", - 0 }, - { "template-name", - 't', - "name", - 0, - "AWS fleet provisioning template name", - 0 }, - { "template-param", - 'p', - "json", - 0, - "[optional] Fleet Prov additional parameters", - 0 }, - { "data-endpoint", 'e', "name", 0, "AWS IoT Core data endpoint", 0 }, - { "root-ca-path", 'r', "path", 0, "Path to key for client certificate", 0 }, - { 0 } -}; +static struct argp_option opts[] + = { { "user-group", + 'u', + "name", + 0, + "[optional]GGL_SYSTEMD_SYSTEM_USER user and group \":\" seprated", + 0 }, + { "claim-key", + 'k', + "path", + 0, + "[optional]Path to key for client claim private certificate", + 0 }, + { "claim-cert", + 'c', + "path", + 0, + "[optional]Path to key for client claim certificate", + 0 }, + { "template-name", + 't', + "name", + 0, + "[optional]AWS fleet provisioning template name", + 0 }, + { "template-param", + 'p', + "json", + 0, + "[optional]Fleet Prov additional parameters", + 0 }, + { "data-endpoint", + 'e', + "name", + 0, + "[optional]AWS IoT Core data endpoint", + 0 }, + { "root-ca-path", + 'r', + "path", + 0, + "[optional]Path to key for client certificate", + 0 }, + { 0 } }; static error_t arg_parser(int key, char *arg, struct argp_state *state) { FleetProvArgs *args = state->input; @@ -69,8 +84,14 @@ static error_t arg_parser(int key, char *arg, struct argp_state *state) { case 'r': args->root_ca_path = arg; break; + case 'u': + args->user_group = arg; + break; case ARGP_KEY_END: - // ALL keys have defaults further in. + if (args->user_group == NULL) { + args->user_group = "ggcore:ggcore"; + } + // All keys are optional other are set down the line break; default: return ARGP_ERR_UNKNOWN; diff --git a/fleet-provisioning/include/fleet-provisioning.h b/fleet-provisioning/include/fleet-provisioning.h index 8d29795e1..16825d3cd 100644 --- a/fleet-provisioning/include/fleet-provisioning.h +++ b/fleet-provisioning/include/fleet-provisioning.h @@ -16,6 +16,7 @@ typedef struct { char *data_endpoint; char *root_ca_path; char *iotcored_path; + char *user_group; } FleetProvArgs; GglError run_fleet_prov(FleetProvArgs *args, pid_t *pid); diff --git a/fleet-provisioning/src/entry.c b/fleet-provisioning/src/entry.c index 32a23f3c1..0161cda7f 100644 --- a/fleet-provisioning/src/entry.c +++ b/fleet-provisioning/src/entry.c @@ -28,6 +28,8 @@ #define MAX_TEMPLATE_PARAM_LEN 4096 #define MAX_PATH_LEN 4096 +GglBuffer ggcredentials_path = GGL_STR("/ggcredentials"); + static GglError start_iotcored(FleetProvArgs *args, pid_t *iotcored_pid) { char *iotcore_d_args[] = { args->iotcored_path, "-n", "iotcoredfleet", "-e", @@ -191,8 +193,8 @@ static GglError fetch_from_db(FleetProvArgs *args) { return GGL_ERR_OK; } -static GglError update_cred_access(void) { - char *args[] = { "chown", "-R", "ggcore:ggcore", "/ggcredentials/", NULL }; +static GglError update_cred_access(char *user_group) { + char *args[] = { "chown", "-R", user_group, "/ggcredentials/", NULL }; GglError ret = ggl_exec_command(args); if (ret != GGL_ERR_OK) { @@ -272,8 +274,6 @@ static GglError update_iot_endpoints(void) { } GglError run_fleet_prov(FleetProvArgs *args, pid_t *pid) { - GglBuffer ggcredentials_path = GGL_STR("/ggcredentials"); - int config_dir; GglError ret = ggl_dir_open(ggcredentials_path, O_RDONLY, false, &config_dir); @@ -401,7 +401,7 @@ GglError run_fleet_prov(FleetProvArgs *args, pid_t *pid) { return ret; } - ret = update_cred_access(); + ret = update_cred_access(args->user_group); if (ret != GGL_ERR_OK) { return ret; } diff --git a/ggl-file/src/file.c b/ggl-file/src/file.c index d24218b47..2f86fbc49 100644 --- a/ggl-file/src/file.c +++ b/ggl-file/src/file.c @@ -488,9 +488,6 @@ GglError ggl_file_read_path_at(int dirfd, GglBuffer path, GglBuffer *content) { int fd; GglError ret = ggl_file_openat(dirfd, path, O_RDONLY, 0, &fd); if (ret != GGL_ERR_OK) { - GGL_LOGD( - "Err %d while opening file: %.*s", errno, (int) path.len, path.data - ); return ret; } GGL_CLEANUP(cleanup_close, fd); @@ -548,5 +545,17 @@ GglError ggl_file_read_path(GglBuffer path, GglBuffer *content) { return GGL_ERR_FAILURE; } GGL_CLEANUP(cleanup_close, base_fd); - return ggl_file_read_path_at(base_fd, rel_path, content); + + GglError ret = ggl_file_read_path_at(base_fd, rel_path, content); + if (ret != GGL_ERR_OK) { + GGL_LOGE( + "Err %d occurred while reading file %.*s", + errno, + (int) rel_path.len, + rel_path.data + ); + return ret; + } + + return GGL_ERR_OK; } diff --git a/ggl-recipe/src/recipe.c b/ggl-recipe/src/recipe.c index df5d47bbe..b442804cc 100644 --- a/ggl-recipe/src/recipe.c +++ b/ggl-recipe/src/recipe.c @@ -5,6 +5,7 @@ #include "ggl/recipe.h" #include #include +#include #include #include #include @@ -316,10 +317,10 @@ static GglError manifest_selection( } } - GglList selection_deafult + GglList selection_default = GGL_LIST(GGL_OBJ_BUF(GGL_STR("all"))); return lifecycle_selection( - &GGL_OBJ_LIST(selection_deafult), + &GGL_OBJ_LIST(selection_default), recipe_map, selected_lifecycle_object ); @@ -485,6 +486,12 @@ GglError ggl_recipe_get_from_file( recipe_dir, GGL_STR("yml"), base_name, &content ); if (ret != GGL_ERR_OK) { + GGL_LOGE( + "Err %d could not open recipe file for: %.*s", + errno, + (int) base_name.buf.len, + base_name.buf.data + ); return ret; } } diff --git a/recipe2unit/src/unit_file_generator.c b/recipe2unit/src/unit_file_generator.c index ac6b10d22..7491f47b2 100644 --- a/recipe2unit/src/unit_file_generator.c +++ b/recipe2unit/src/unit_file_generator.c @@ -286,7 +286,7 @@ static GglError update_unit_file_buffer( return ret; } } else { - // The deafult timeout is 120 seconds + // The default timeout is 120 seconds ret = ggl_byte_vec_append(out, GGL_STR("TimeoutSec=120\n")); if (ret != GGL_ERR_OK) { return ret; diff --git a/tes-serverd/src/http_server.c b/tes-serverd/src/http_server.c index 617b74793..c800bcf64 100644 --- a/tes-serverd/src/http_server.c +++ b/tes-serverd/src/http_server.c @@ -161,7 +161,7 @@ static void request_handler(struct evhttp_request *req, void *arg) { evbuffer_free(buf); } -static void deafult_handler(struct evhttp_request *req, void *arg) { +static void default_handler(struct evhttp_request *req, void *arg) { (void) arg; GglBuffer response_cred_buffer @@ -205,7 +205,7 @@ GglError http_server(void) { evhttp_set_cb( http, "/2016-11-01/credentialprovider/", request_handler, NULL ); - evhttp_set_gencb(http, deafult_handler, NULL); + evhttp_set_gencb(http, default_handler, NULL); // Bind to available port handle = evhttp_bind_socket_with_handle(http, "0.0.0.0", 0);