Skip to content

Home

Jump to bottom
bmpotter edited this page Mar 6, 2018 · 44 revisions

Horizon - Anax

Horizon enables a "Programmable Edge" for Analytics and Cognitive Applications. See http://bluehorizon.network/ for more information. Anax is the core component of horizon, running on each system participating in horizon. This wiki page provides information about using specific aspects of anax.

APIs

Anax can run in 2 modes, or both at the same time, and there is an API for each:

Agreements

Analytic/cognitive applications are sent to edge nodes as a result of an agreement made between a data producer (normally an edge nodes) and a data consumer (normally an author of an analytic/cognitive application that is "served" by an agbot). Here is some information about the agreement process:

Horizon Agent Attributes

Attributes can be set for the edge node as a whole, or for specific microservices using the POST /attribute edge API.

Microservices

Horizon deploys and manages microservices on edge nodes. Microservices are docker containers the provide data and other services to workloads.

Managed Workloads

Horizon manages the lifecycle, connectivity, and resource caps of analytic/cognitive applications (workloads) it launches on edge nodes. Some details:

Overriding the Edge Node ID

The node id is the unique identifier of the edge node that is normally generated automatically by horizon (from, for example, the machine serial number). There are a few specific circumstances in which is it useful for the edge node owner to be able to override the generation of the node id. This can be accomplished in 3 different ways:

  • Set CMTN_DEVICE_ID in /etc/default/horizon before starting the horizon service
  • Specify the node id to the hzn register command via the -n flag.
  • If registering the node via the API, specify the node id in the POST /node method.

Importing/Exporting Ethereum Credentials

Ethereum is the default blockchain used by horizon to record agreements between producers (edge nodes) and consumers (agreement bots/workload authors). By default a new set of ethereum credentials is automatically created for each edge node. But if currency will ultimately be exchanged for data produced (via horizon metering - see next section), it can be useful for a data producer to use the same ethereum credentials on all of the edge nodes he/she owns. This can be accomplished by letting horizon create the ethereum credentials on your first edge node and then export/import them to all of your other edge nodes:

  1. Export (capture) the ethereum credentials from your first edge node by downloading and running ethcap.sh:
    • cd /tmp
    • COMMON=/var/snap/bluehorizon/common /snap/bluehorizon/<revision>/scripts/anax/ethcap.sh
  2. The command above produces a file called ethcreds.tar.gz that contains the credentials. Copy this file to the /tmp directory of your other edge nodes.
  3. Import (restore) the ethereum credentials on your other nodes by downloading and running ethrestore.sh before registering the node and services:
    • cd /tmp
    • COMMON=/var/snap/bluehorizon/common /snap/bluehorizon/<revision>/scripts/anax/ethrestore.sh ethcreds.tar.gz
  4. You can verify that the ethereum credentials are now the same by running the Horizon edge node API on all of your edge nodes:
    • curl -sS http://localhost/status | jq .geth

Note* the above command shows the "eth_balance" as an empty array before the ethereum container (geth) is up and running (geth starts running after the node and service registration).

Metering Records for Agreements

Metering records for agreements can be generated by horizon to record (ultimately on the blockchain) the amount/value of data a producer has given to a consumer. This can be used as the basis for value or currency exchange between the consumer and producer. The consumer should use the metering records in the blockchain and in its agbot to convert metering tokens into some value or currency that is transferred to the producer outside the Horizon platform. The steps required to read metering records from the blockchain and agbot APIs are documented in Reading Metering Records from the Blockchain. An overview of the steps to use metering records follows:

  1. Specify that metering records should be generated as part of an agreement, either in the edge node, the agbot policy file, or both. See Enabling Metering
  2. Ssh to your agbot and use the Agreement Bot API to find archived agreements that you want the metering records for.
    • For example, to find the archived agreements for a particular node: curl -sS http://localhost:8046/agreement | jq '.agreements.archived[] | select(.device_id =="<device-id>") | {device_id,policy_name,current_agreement_id,agreement_creation_time,terminated_description,metering_tokens} '
  3. Follow the steps in Reading Metering Records to read the metering records from the blockchain, and convert those metering records to a specific value that can be transferred to the producer.
  4. Optionally delete agreements from the agbot that have been processed.

Agreement Protocols

An agreement protocol is used by Horizon to define the mechanics of how an agreement is made and maintained within the Horizon platform. Horizon supports a choice of agreement protocols. The "Citizen Scientist" protocol is the default and it uses an ethereum blockchain to record agreements and metering. The "Basic" protocol is a simpler protocol that does not depend on a blockchain, but it shouldn't be used in an unknown/untrusted context. Both Nodes and Agreement Bots can specify the use of a given agreement protocol.

In the policy file of an agreement bot, the agreement protocol choice is specified in the agreementProtocols section:

    "agreementProtocols": [
        {
          "name": "Basic"
        }
    ]

On a node, each service can be registered with an agreement protocol by placing the chosen agreement protocol within the attributes list of the POST /service API:

    "attributes": [
    ...
        {
          "id": "agreementprotocol",
          "short_type": "agreementprotocol",
          "label": "Agreement Protocol",
          "publishable": true,
          "mappings": {
            "protocols": [
              {"Basic":[]}
            ]
          }
        },
    ...
    ]

Registering Blockchain Instances

The Horizon Exchange has APIs that allow an authenticated user to register a blockchain client instance. The registered instance can be referred to by edge node and agreement bot policies that use the Citizen Scientist agreement protocol. See the next section for details on how a blockchain client container instance can be used by the agreement protocol.

The Exchange API ( PUT /v1/bctypes/{bctype}/blockchains/{name} ) stores metadata about the blockchain client container in the details field. The details field is a stringified JSON document that describes 2 main piece of information; (a) container image, and (b) instance configuration.

The container image metadata in the "deployment_description" field is similar to the container image metadata needed to run a workload container. It is used to download and verify the container image and the deployment pattern in which the container should run. The instance configuration in the "instance" field configures the blockchain client container by passing in the specified fields as environment variables to the container. The following is the schema of the body of the Exchange PUT API:

{
  "chains": [
    "arch": <hardware_architecture>,
    "deployment_description": {
      "deployment": <service deployment string>,
      "deployment_signature": <signature of the service deployment string>,
      "deployment_user_info": <extra deployment config string>,
      "torrent": {<torrent metadata>}
    },
    "instance": {
      "blocksURLs": <a list of white space seperated IP addresses where blockchain blocks can be found, no default>,
      "chainDataDir": <path in the Ananx filesystem where the blocks will reside, no default>,
      "discoveryURLs": <a list of white space seperated URLs where blockchain network information can be downloaded, no default>,
      "port": <the port where ethereum listens for peers (not the rpc port), defaults to 33303>,
      "hostname": <host name passed in when starting ethereum, defaults to first qualified of machine host name>,
      "identity": <identity passed in to ethereum, defaults to machine architecture>,
      "kdf": <ethereum KDF parameter value, defaults to --lightkdf>,
      "pingHost": <the host to ping to determine if network connectivity exists, no default>,
      "colonusDir": <the path in the Anax filesystem where blockchain identity is stored, defaults to /root/eth>,
      "ethDir": <the path in the Anax filesystem where ethereum runtime state is stored, defaults to /root/.ethereum>,
      "maxPeers": <passed to the maxPeers parameter when starting ethereum, defaults to 12>,
      "gethLog": <not used anymore, defaults to /tmp/geth.log>,
    }
  ]
}

If a blockchain client container is expected to run with other blockchain container instances on the same machine, then it is important that the "chainDataDir", "port", "identity", "colonusDir", "ethDir" fields are unique across all blockchains of the same type.

Multiple Blockchains

Anax supports the ability for a producer and a consumer to agree on the use of a specific blockchain as part of the agreement protocol. You can read about this feature here.

Clone this wiki locally