THNGHUB

Documentation for THNGHUB versions:

• 5.1.2 for Node 4 LTS
• 6.1.2 for Node 6 LTS

Overview

THNGHUB is our Local Cloud Gateway. In short - THNGHUB is a local extension of the EVRYTHNG Platform. It is a lightweight modular software application focusing on two main goals:

• First, it lets you integrate non-IP protocols (e.g., Zigbee, Bluetooth, DALI, etc.) to the Web, acting as a gateway to the EVRYTHNG platform.
• Then, it allows you to locally control connected products when connected to a local network, leading to lower latency and cloud independence.

What this means is that via THNGHUB non-Internet devices get a Web API with the EVRYTHNG Platform (good for control while out-of-home, good for analytics and integrations), as well as a local API (better in terms of latency, works when the Internet is not working). Additionally, THNGHUB maintains consistency between the cloud and the local API with a powerful synchronisation module.

To read more about the unique features of THNGHUB check out our blogpost, watch the introductory video below or download our product sheet.

📘

Note

THNGHUB is currently only available as an enterprise feature upon request. If you'd like to use it for your projects please contact us.

🚧

Important

THNGHUB will store updates to Platform resources locally and upload them to the cloud API at the earliest opportunity. This means that if more than the configured number of updates per resource type are stored when the hub is offline, new items will begin to replace old ones, in a first in, first out manner. If the hub is connected to the internet, these updates will have already been synchronised.

Hardware Requirements

THNGHUB is a modular Node.js application designed to work on typical Linux gateways. It can run on many hardware platforms such as the BeagleBone Black, the Intel Edison, or the Raspberry Pi (from B+ onwards).

The minimal recommended configuration is as follow:

Clock speed: >= 700 MHz (>=ARMv6 or x86)
Cores: >= 1
Flash: >= 512 MB
RAM: >= 256MB

Software Requirements

THNGHUB is meant for Linux environments, runs on Node.js, and uses ZeroMQ.

Installing THNGHUB (on Linux)

Software Installation

• Install Node.js version >= 4.2.x
• Install ZeroMQ: sudo apt-get install libzmq-dev
• Log into NPM (private repo, contact us first): npm login
• Install THNGHUB: npm install --production --no-optional -g @evrythng/thng-hub (the --production and --no-optional options are important to minimize installation time)
• or update THNGHUB: npm update -g @evrythng/thng-hub

Linux Configuration

For optimal performance the following configuration needs to be applied on Linux:

In /etc/sysctl.conf add/change:

net.core.somaxconn=2048
net.ipv4.tcp_fin_timeout=2

then run sysctl -p.

Configuration

When running THNGHUB you should specify a configuration JSON file as a command line parameter. This needs to be a JSON file that contains your configuration options: thng-hub --config ~/path/to/config.json

Or if you don't have thng-hub installed globally: node bin/thng-hub --config ~/path/to/config.json

Here is a sample config file. Remember to replace apiKey and hubThngId with your own key and Thng ID values:

{
  "apiUrl": "https://api.evrythng.com",
  "pushUrl": "mqtts://mqtt.evrythng.com:443/mqtt",
  "apiKey": "$TRUSTED_APPLICATION_API_KEY",
  "hubThngId": "$HUB_THNG_ID",
}

{
  "target": "prod",
  "apiKey": "$TRUSTED_APPLICATION_API_KEY",
  "hubThngId": "$HUB_THNG_ID",
}

All the configuration options can be overridden using command line arguments or environment variables. For example, if you wanted to use a different apiKey to the one specified in the config file you could do:

thng-hub --config ~/path/to/config.json --apiKey $TRUSTED_APPLICATION_API_KEY

Or:

apiKey=$TRUSTED_APPLICATION_API_KEY thng-hub --config ~/path/to/config.json

Note that command line arguments take precedence over environment variables.

Main Configuration Options

• apiUrl - String, Required, v6.x only
  The URL to the EVRYTHNG API the hub will use. This is usually https://api.evrythng.com.

• pushUrl - String, Required, v6.x only
  The URL to the Pub/sub broker (THNGPUSH). This is usually mqtts://mqtt.evrythng.com:443/mqtt.

• target - String, Required, v5.x only
  The target environment. This is usually prod.

• apiKey - String, Required
  The EVRYTHNG API key that THNGHUB will use to connect to the cloud (should be a Trusted Application API key).

• hubThngId - String, Required
  The Thng ID corresponding to the hub in the EVRYTHNG platform.

• security.request - Boolean
  Default: true
  Is incoming data expected to be encrypted (e.g., Actions)?

• security.response - Boolean
  Default: false
  Is outgoing data expected to be encrypted (e.g., Properties updates)? Setting this to true will have an important negative impact on performance and is only advised if the local network is not secure (e.g., not using WPA). The format of the configuration options is as follows:

  "security" : {
    "request": true,
    "response": false
   }

• noLogs - Boolean, Not applicable for THNGHUB v6.x
  Default: false
  Disable all logging.

• enableConsoleLogging - Boolean, Not applicable for THNGHUB v6.x
  Default: false
  Output logs to the console (do not use in production as this will impact performance).

• doNotUseProcessIdInName -Boolean, v6.x only
  Do not use the process ID in the log name.

• longitude - Integer
  The longitude for THNGHUB, between -180.0 to 180.0.

• latitude - Integer
  The latitude for THNGHUB, between between -90.0 to 90.0.

• resourcesManagedPerMQTTClient - Integer
  Default: 50
  Sets the number of resources (action types, Thngs and collections) managed per MQTT client.

• useHttpForCloudUpdates - Boolean
  Default: false
  Sends data to the cloud via HTTP rather than MQTT.

• firmwareDownloadDirectory - String
  Default: ./firmware
  Where firmware updates should be stored on the local filesystem. If the string begins with a ./ the hub assumes this as relative to the root of the hub directory. For example, if the hub is installed in the /usr/local/hub directory then, by default, firmware updates are saved to /usr/local/hub/firmware. If the string does not begin with a ./ then the path is considered absolute.

Advanced Configuration Options

• syncPolicy.messageLimit - Integer
  Default: 20
  Maximum number of messages that will be aggregated before being sent to the API in one call.

• syncPolicy.maxRetries - Integer
  Default: 3
  Number of times the REST client will retry delivering a message on failure, before pausing.

• syncPolicy.shortWait - Integer
  Default: 5
  Time in milliseconds the REST client will wait between each delivery attempt on failure.

• syncPolicy.longWait - Integer
  Default: 1000
  Time in milliseconds the REST client will pause after the maximum number of short waits is reached.

• syncPolicy.queueSize - Integer
  Default: 20000
  Maximum number of messages present in the synchronization queue at any one time. Trade memory for data integrity.

• syncPolicy.parallelMode - Boolean
  Default: false
  Enables parallel delivery of messages in the synchronization queue. Trade memory for data integrity.

• maxResourceSizes.thngs - Integer
  The maximum number of Thngs the hub will store locally. Once the hub reaches this limit it will prevent any new things from being discovered and return an error to the DAL.

• maxResourceSizes.actionTypes - Integer
  The maximum number of action types the hub will store locally. Once the hub reaches this limit it will not subscribe to any new action types and log an error.

• maxResourceSizes.collections - Integer
  The maximum number of collections the hub will store locally. Once the hub reaches this limit it will not subscribe to any new collections and log an error.

• maxResourceSizes.properties - Integer
  The maximum number of properties per thng and property key the hub will store locally. Once the hub reaches this limit it will overwrite old properties with new ones.

• maxResourceSizes.actions - Integer
  The maximum number of actions per thng and action type or per collection and action type the hub will store locally. Once the hub reaches this limit it will overwrite old actions with new ones.

Logging Configuration Options

THNGHUB provides many tunable options for logging, note that some options will have an important effect on performance at scale.

The logger can have different transports defined, which are essentially the storage devices for the log. THNGHUB is currently setup to send data to the console (if enabled) and to a file.

General Logging Options

These options are only applicable for THNGHUB v6.x

• logger.level - String
  Default: "error"
  The level of logging messages thng-hub will output. Can be one of "trace", "debug", "info", "warn", "error" or "fatal".

• logger.file - Boolean
  Default: true
  If true will log to a file.

• logger.console - Boolean
  Default: false
  If true will log to the console.

• logger.path - String
  Default: "./logs/thng-hub.log"
  Path to a file where the log messages will be stored on the file system.

• logger.period - String
  Default "1d"
  The period at which to rotate the logs. See here for the different string options .

• logger.count -Integer
  Default: 5
  Max number of log files to keep.

Console Logging Options

These options are not applicable for THNGHUB v6.x.

• logger.transport.Console.level - String
  Default: debug
  Level of messages for this transport. Options are debug, warn, info.

• logger.transport.Console.json - Boolean
  Default: false
  Flag indicating whether or not the output should be JSON. If true, will log out multi-line JSON objects.

• logger.transport.Console.colorize - Boolean
  Default: true
  Flag indicating if we should colorize output.

• logger.transport.Console.timestamp - Boolean
  Default: true
  Flag indicating if we should prepend output with timestamps.

• logger.transport.Console.prettyPrint - Boolean
  Default: true
  Indicating if we should util.inspect the metadata. If function is specified, its return value will be the string representing the metadata.

File Logging Options

These options are not applicable for THNGHUB v6.x.

• logger.transport.File.level - String
  Default: info
  Level of messages for this transport. Options are debug, warn, info.

• logger.transport.File.filename - String
  Default: ./logs/thng-hub.log
  Filename of the log file to write output to.

• logger.transport.File.json - Boolean
  Default: false
  Flag indicating whether or not the output should be JSON. If true, will log out multi-line JSON objects.

• logger.transport.File.maxsize - Integer
  Default: 5242880
  Max size in bytes of the logfile, if the size is exceeded then a new file is created, a counter will become a suffix of the log file.

• logger.transport.File.maxFiles - Integer
  Default: 5
  Limit the number of files created when the size of the logfile is exceeded.

• logger.transport.File.colorize - Boolean
  Default: false
  Flag indicating if we should colorize output.

• logger.transport.File.timestamp - Boolean
  Default: true
  Flag indicating if we should prepend output with timestamps.

• logger.transport.File.prettyPrint - Boolean
  Default: true
  If true, additional JSON metadata objects that are added to logging string messages will be displayed as a JSON string representation. If function is specified, its return value will be the string representing the metadata.

• logger.transport.File.tailable - Boolean
  Default: true
  If true, log files will be rolled based on maxsize and maxfiles, but in ascending order. The filename will always have the most recent log lines. The larger the appended number, the older the log file. This option requires maxFiles to be set, or it will be ignored.

Local APIs

THNGHUB offers a local REST and Pub/Sub API (via WebSockets or MQTT). The resources available represent a subset of the EVRYTHNG resources. All other resources are available via the API.

API Authentication

THNGHUB features an authorization mechanism in sync with the EVRYTHNG Platform. As a consequence you need to make sure you use an API key authorized (either via the Authorization HTTP header or the access_token query parameter) to perform any local operation just like when interacting with the REST API. For more information, please refer to the API Key Scopes and Permissions page.

📘

Notes

• Authentication is cached locally and cache expires only if the hub is online.
  • Scope is managed at resource level, i.e. if key is allowed to see the Thng / collection / action type it is allowed to access the properties and the actions. This is the main difference with the EVRYTHNG Platform API.
  • Creation of properties / actions is forwarded to platform with the same key that was used to make the request.

Host and Ports

THNGHUB APIs are accessible by using the local IP address of the hub. Ports can be configured in the config file but by default, the following ports are assigned:

HTTP (REST): 8787
MQTT: 4001
WebSockets: 4000

Resources

Please note that all HTTP resources conform to the Platform's pagination implementation.

• /thngs - HTTP: ✓ Pub/Sub: ✓ CRUD: R
  The details for all the Thngs managed by this Hub, all the hubs and name, description and IP of Thngs managed by other hubs.

• /thngs/:thngId - HTTP: ✓ Pub/Sub: ✓ CRUD: R

• /thngs/:thngId/properties - HTTP: ✓ Pub/Sub: ✓ CRUD: C R U

• /thngs/:thngId/properties/:key - HTTP: ✓ Pub/Sub: ✓ CRUD: R U

• /actions - HTTP: ✓ Pub/Sub: ✓ CRUD: R

• /thngs/:thngId/actions/all - HTTP: ✓ Pub/Sub: ✓ CRUD: C R

• /thngs/:thngId/actions/all/:actionId - HTTP: ✓ Pub/Sub: ✖ CRUD: R

• /thngs/:thngId/actions/:actionType - HTTP: ✓ Pub/Sub: ✓ CRUD: C R

• /thngs/:thngId/actions/:actionType/:actionId - HTTP: ✓ Pub/Sub: ✖ CRUD: R

• /collections/ - HTTP: ✓ Pub/Sub: ✖ CRUD: R

• /collections/:collectionId - HTTP: ✓ Pub/Sub: ✓ CRUD: R

• /collections/:collectionId/thngs - HTTP: ✓ Pub/Sub: ✓ CRUD: R
  Changes in the list of Thngs contained in the collection. For this topic the messages received by the subscriber have the following structure:

{
    "operation": <operation>,
    "timestamp": <timestamp>,
    "resources": [ <thngId>, ... ]
}

where <operation> is a value from: thngsAdded, thngsRemoved, thngsCleared (all Thngs were removed).

• /collections/:collectionId/actions/all - HTTP: ✓ Pub/Sub: ✓ CRUD: C R

• /collections/:collectionId/actions/:actionType - HTTP: ✓ Pub/Sub: ✓ CRUD: C R

• /collections/:collectionId/actions/:actionType/:actionId - HTTP: ✓ Pub/Sub: ✖ CRUD: R

Pub/Sub Options

The following options are supported when subscribing via MQTT or WebSockets. Options are passed in query parameters, like in URIs.

• project - Ref
  Just like with our HTTP API, it scopes the subscription on the given project.

• pubStates - Integer
  If set to 1, clients receive immediately an initial notification containing the last value of the topic that was directly published by the server (i.e., latest property, location or action value).

Metadata for the Hub's Thng

On startup THNGHUB will update the Thng corresponding to your Hub as follows (available both remotely and locally):

• add a customField with the THNGHUB version e.g. {thnghubversion: 3.0.0}
• add a customField with a managedThngs key that points to a collection id e.g, {"managedThngs": "UFaEeKSwGntcyafNTpM4Pepp"}. This collection is created by the hub on startup and will contain all the Thngs that it is managing
• add a property with a name of ~connected indicating if the hub is connected to the cloud
• add a property with a name of latency indicating the latency (in milliseconds) of properties and actions between the time they are created on the hub and the time they’re received by the EVRYTHNG platform
• add a location field with the latitude and longitude of the hub (the hub will use the lat/long specified on startup otherwise this will be determined by the hub’s external IP address)

SDKs and Tools

Local interaction with THNGHUB is supported by all our SDKs by configuring the SDKs with a local THNGHUB address instead of using the cloud.

However, a special plugin for evrythng.js called evrythng-hub.js allows JavaScript developers to use THNGHUB transparently: the plugin automatically runs local call when in the presence of a local THNGHUB and falls back to the cloud when no Hub is present.

Connected Web App

A generic example web app is available to serve as a basis for building apps that communicate with Thngs in the EVRYTHNG Platform. The interface is geared towards allowing you to manage connected devices that may have many properties, which can be updated from the app itself.

You can use this app to view a THNGHUB network and manage devices, and is open-source for free customisation. See the evrythng-connected-app repository for more information.

Local Rules Engine (Reactor)

You can write custom Javascript rules that THNGHUB will download and store. They will be run under the following conditions:

• A property of a Thng that is visible to THNGHUB is updated.
• An action is created on a Thng that is visible to THNGHUB.
• An action is created on a collection that is visible to THNGHUB.

How to install scripts

• Ensure you’ve provided a hubThngId when starting THNGHUB (see above).
• Ensure this Thng is part of a collection (or add it to one).
• Update the collection’s customFields property with a hubrules key and an array of urls that you want THNGHUB to download. For example:

curl -i -H "Content-Type: application/json" \
  -H "Authorization: $OPERATOR_API_KEY" \
  -X PUT 'https://api.evrythng.com/collections/:collectionId' \
  -d '{
    "customFields": {
      "hubrules": [ "https://www.my-hub-rules.com/rule1.js" ]
    }
  }'

• THNGHUB will download these scripts from the url specified and store them for future use.
• Note that if you want to add more rules to a collection make sure that when you update the hubRules key you include the urls already listed in that key otherwise THNGHUB will remove them.
• To remove a rule simply update the hubRules key and exclude the URL for the script you no longer wish to run.
• To update a rule remove it first by updating hubRules and then add it again.
• You can add the hubThngId to multiple collections and if they contain a hubRules key THNGHUB will download the scripts for all of them.
• When THNGHUB updates the scripts it has stored it will update the rulesinstalled property of the hubThngId which will list all the scripts THNGHUB has installed.

Reading Local Rules Engine Logs

The log output of the local Reactor is visible in the standard output console and/or log file output depending on the options set in the hub configuration. See the Logging Configuration Options section for more information on log configuration.

Rules are executed as JavaScript inside a child process. They are run under the conditions outlined above.

Rule Triggered by Property Update

When a property update occurs the following function will be called inside your script:

onThngPropertiesChanged({
    thng: ThngDocument,
    properties: [ PropertyDocument, ... ]
});

As you can see the function is called with an object that contains the Thng on which the property update occurred and the properties updated.

Rule Triggered by Action Creation

When an action is created the following function will be called inside your script:

onActionCreated({
    action: ActionDocument
});

As you can see the function is called with the action that was created.

🚧

Export Functions

Both onActionCreated and onThngPropertiesChanged functions need to be exported via the exports object. Example:

JavaScript

exports.onThngPropertiesChanged = onThngPropertiesChanged;

Updating Rules

Updating a rule requires removing a the old version of the rule from the hubRules customField of the collection.

Rule Utilities

The rules engine makes the following utilities available:

• The console via the console variable.
• The API key you used to start THNGHUB via the apiKey variable.
• The evrythng-extended.js SDK via the EVT variable.
• All of the core modules Node provides.

🚧

Important

THNGHUB rules are running in a number separate processes (one for each rule) but not in complete isolation. As a consequence one rule can influence the performance of another rule as well as the performance of THNGHUB overall.

As a consequence rules should be tested carefully before deploying them to THNGHUB.

Examples

const trustedApp = new EVT.TrustedApp(apiKey);

// called when a Property on a Thng is changed
function onThngPropertiesChanged(event) { 
  console.log('Running onThngPropertiesChanged');
  
  trustedApp.thng(event.thng.id).action('_custom').create({
    customFields: {
      test: true
    }
  });
}

// called when a new Action is created
function onActionCreated(event) { 
  console.log('Running onActionCreated');
  
  if (event.action.thng) {
    trustedApp.thng(event.action.thng).property().update([{
      key: 'testrules2',
      value: true
    }]);
  } else if (event.action.collection) {
    trustedApp.collection(event.action.collection).update({
      customFields: {
        testrules: true
      }
    });
  }
}

// handler functions have to be exported!
exports.onThngPropertiesChanged = onThngPropertiesChanged; 
exports.onActionCreated = onActionCreated;

Local Encryption

In order to allow local APIs to be secure, THNGHUB supports the encryption of JSON payloads and API keys. This system is implemented based on a secret exchanged via the cloud (i.e., over a secure TLS channel) used to encrypt payloads and API keys.

Local encryption is activated by default for inbound traffic (from clients to the hub), however not for outbound traffic (from hub to clients), that is: responses (REST) as well as updates (pub/sub) are not encrypted. You can force the encryption of responses and updates by setting security.encryptResponses to true.

🚧

Important

Setting security.encryptResponses to true will have an important negative impact on the performance of THNGHUB and is only advised if the local wireless network is not secure (i.e., not using WPA) and for deployments where a hub manages a limited number of Thngs (e.g., in a home environment).

Working With Encryption

The best way to work with encryption is by using evrythng-hub.js as the library deals with the secret message setup as well as the encryption/decryption of requests and responses for the MQTT, WS and HTTP APIs.

If you do not use evrythng-hub.js you will have to implement the encryption (and decryption if the security.response flag is active) of requests and responses in your own code. The THNGHUB local encryption system uses JSON Web tokens encrypted with the JWE specification.

The algorithm used for encryption is AES GCM Key Wrap. The symmetric key used for key encryption is serialized into the Thng which represents the Hub, into the key custom field. If the key is not present it is automatically generated by the Hub at startup. If no hubThngId is provided in the configuration, encryption is not possible and requests using an encrypted key fail.

Step by Step Encryption Process for POST / PUT / Pub Operations

• Prepare the jwt claims :

{
    "iss" : "request apiKey in plain text",
    "aud" : "target hubThngId",
    "jti" : "Unique ID for this request. Cannot be reused to prevent replay attacks.",
    "sub" : "target url. Must match the request resource (e.g., /thngs/)",
    "data" : {
      <actual payload of the request>
    }
}
  ```
* Sign this JWT with the `kid` of the key. This is the secret shared via the cloud and in a custom field called "key" in the Thng corresponding to the THNGHUB instance.
* Encrypt the resulting jwt into a JSON serialized JWE using the key available in the hub Thng.
* Encrypt the `apiKey` into a compact serialized JWE using the key available in the hub Thng.
* Send the resulting JSON as a normal request (POST/PUT) to the hub API, along with the resulting Compact JWE token in the `Authorization` HTTP header.


### Request Process for GET / Sub operations

The `Authorization` HTTP header should be a valid encrypted EVRYTHNG API key serialized into a 
JWE Compact String.


### Step by Step Decryption Process

* Decrypt the JWE using the _AES GCM_ key wrapping algorithm, and the symmetric key serialized into using the `key` custom field.
* The resulting clear text is a JWT token to be validated using the `key.kid` secret.
* Validate the JWT and check its claims as explained here above
* Use the `data` claim as the response payload.


### Response validation

In this case, the response to a request made with an encrypted API key is also a JSON Web token encrypted with JWE. It is the responsibility of the Client to check the validity of the JWT claims.

In particular that the:

* `iss` field contains the id of the Thng representing the hub.
* `aud` field contains the API key used to make the request.
* `sub` field contains the url of the request made.
* `jti` field contains a message identifier value used for the first time (check this to avoid replay attacks).
* `data` contains the payload of the response.


### Encryption Playground

The encryption playground allows you to encrypt and decrypt API keys and payloads. It is an HTML file located in the `tools/encryption/` folder of THNGHUB source code (not available in the NPM version).


### Useful libraries

A number of client libraries for many programming languages are available on [JWT.io](https://jwt.io/).
___


## Distributed Hubs

To allow deployments with more than one hub, THNGHUB implements a local distribution system that lets a client discover all the hubs in a deployment and all the Thngs they manage. Based on this information, the client can locally contact each Thng on the hub it is connected to, within the same project. All THNGHUB instances use this distributed model, which cannot be disabled. 

This means that when a hub starts up:

1. It will use its Application API Key to search for a collection with the tag `HubDistribution` in the project.
2. If the collection exists, the hub will add itself to the collection. 
3. If the collection does not exist, the Hub will create the collection automatically and add a `HubDistribution` tag.

[block:callout]
{
"type": "info",
"body": "The `apiKey` provided in the hub configuration must have a read/write access to the corresponding Project.",
"title": "Note"
}
[/block]

The hub will automatically be subscribed to MQTT messages about that collection. This means that when a hub is added or removed from the collection, all other hubs in the collection will be notified.

Note that if the `HubDistribution` tagged collection is accidentally deleted then the distribution group will be lost and the Hubs will only work in local mode until they are restarted. Upon restart, the first hub to try to connect to the Platform will recreate the collection.

[block:callout]
{
"type": "warning",
"title": "Attention",
"body": "In order to communicate with each other, distributed hubs use the `customField` `ipAddress`. The value of this field is set by the hub itself."
}
[/block]

### Distributed Thng Model

All Thngs managed by a hub are maintained with a `~managedThngs` customField. This field is only available locally. To list all Thngs (including all the ones managed by all distributed hubs on local network), send a `GET` request or subscribe (via MQTT or WS) to `/thngs` endpoint. This returns all Thngs and Hubs in the local distributed network based on the model below. Updates of IP, name and description are propagated via the `/thngs` resource of all hubs in the distribution.

[block:code]
{
"codes": [
  {
    "code": "DistributedThngDocument = {\n  \"id\": Ref,\n  \"name\": String,\n  \"description\": String,\n  \"customFields\": {\n    \"host\": ipAddress,\n    \"hubThngId\": distributedHubThngId\n  }\n}",
    "language": "json",
    "name": "Distributed thng model"
  }
]
}
[/block]

where `ipAddress` - address to distributed hub (format: `"ipv4":"pubsubPort"`) provides a way to subscribe directly to the Thngs managed by a distributed hub via the `mqtt` protocol.

As an example this snippet is the result of  a `GET` `/thngs` (or a subscription) in a cluster of 2 hubs each managing a number of Thngs: 

[block:code]
{
"codes": [
  {
    "code": "[\n  {\n    \"id\": \"UhTbrmdHcednMAwe3BbKYrEq\",\n    \"createdAt\": 1461836090220,\n    \"updatedAt\": 1464022637887,\n    \"name\": \"Thng Hub 1\",\n    \"description\": \"Test Hub 1 Description\",\n    \"product\": \"UhTb7F5sTVAn6UwB3exKYNec\",\n    \"customFields\": {\n      \"ipAddress\": {\n        \"v4\": \"192.168.0.239:4001\",\n        \"v6\": \"fe80::3e15:c2ff:fec8:d5a2:4001\"\n      },\n      \"key\": {\n        \"alg\": \"A256GCM\",\n        \"k\": \"i51nq0Ltn9ij59Be58-ZI2KLD3v1BNqNxvGw3B_nzdA\",\n        \"kid\": \"TmRJVtmz3wrMrU8-F4arI_9Ht8lQ-VhFmGg5XjToqD4\",\n        \"kty\": \"oct\",\n        \"use\": \"enc\"\n      },\n      \"nodeversion\": \"v4.2.2\",\n      \"osarch\": \"x64\",\n      \"ostype\": \"Darwin\",\n      \"thnghubversion\": \"3.1.1\",\n      \"~managedThngs\": [\n        {\n          \"id\": \"UhTbrmdHcednMAwe3BbKYrEq\",\n          \"name\": \"Thng Hub 1\"\n        },\n        {\n          \"id\": \"UYb7t8B2c9PfAkB4CC6Fdfcq\",\n          \"name\": \"Bulb 1-1\"\n        },\n        {\n          \"id\": \"UEx7tsW5tBVs79eqnheMmr7q\",\n          \"name\": \"Bulb 1-2\"\n        },\n        {\n          \"id\": \"UhSN98ExytsCdkeHWCMFdCes\",\n          \"name\": \"Bulb 1-3\"\n        }\n      ]\n    },\n    \"collections\": [\n      \"UETbr8qpQGmpHawB3XA3Drpg\"\n    ],\n    \"properties\": {\n      \"~connected\": true\n    }\n  },\n  {\n    \"id\": \"Uhyx7GKtcBAHMUwe3BSpYrqd\",\n    \"name\": \"Thng Hub 2\",\n    \"customFields\": {\n      \"host\": \"192.168.0.239:4008\",\n      \"hubThngId\": \"Uhyx7GKtcBAHMUwe3BSpYrqd\"\n    }\n  },\n  {\n    \"id\": \"UEx7tsW5tBVs79eqnheMmr7q\",\n    \"createdAt\": 1461583451336,\n    \"updatedAt\": 1463500091489,\n    \"name\": \"Bulb 1-2\",\n    \"properties\": {\n      \"bulb\": \"Bulb 1-2\"\n    }\n  },\n  {\n    \"id\": \"UhSN98ExytsCdkeHWCMFdCes\",\n    \"createdAt\": 1461583473886,\n    \"updatedAt\": 1463499153748,\n    \"name\": \"Bulb 1-3\",\n    \"properties\": {\n      \"bulb\": \"Bulb 1-3\"\n    }\n  },\n  {\n    \"id\": \"UYb7t8B2c9PfAkB4CC6Fdfcq\",\n    \"createdAt\": 1461583432283,\n    \"updatedAt\": 1464019232189,\n    \"name\": \"Bulb 1-1\",\n    \"identifiers\": {\n      \"mac\": \"1234123413243232\"\n    },\n    \"properties\": {\n      \"power\": 110,\n      \"temperature\": 26\n    },\n    \"tags\": [\n      \"testHub\"\n    ],\n    \"customFields\": {\n      \"model\": \"123ab2\"\n    }\n  },\n  {\n    \"id\": \"UYxrtPPn9BeP79eq4EVM3sVc\",\n    \"name\": \"Bulb 2-1\",\n    \"customFields\": {\n      \"host\": \"192.168.0.239:4008\",\n      \"hubThngId\": \"Uhyx7GKtcBAHMUwe3BSpYrqd\"\n    }\n  },\n  {\n    \"id\": \"UYbN9P9XcQPWAFe4fWMkdfHm\",\n    \"name\": \"Bulb 2-2\",\n    \"customFields\": {\n      \"host\": \"192.168.0.239:4008\",\n      \"hubThngId\": \"Uhyx7GKtcBAHMUwe3BSpYrqd\"\n    }\n  },\n  {\n    \"id\": \"UYbr9taf9eVsr9BqnEB6Gsfp\",\n    \"name\": \"Bulb 2-3\",\n    \"customFields\": {\n      \"host\": \"192.168.0.239:4008\",\n      \"hubThngId\": \"Uhyx7GKtcBAHMUwe3BSpYrqd\"\n    }\n  }\n]",
    "language": "json"
  }
]
}
[/block]

___


## Developing Plugins for Other Protocols

### Device Abstraction Layer (DAL)

Plugins are applications that bridge a proprietary or non-IP protocol (e.g., Zigbee, Bluetooth, PLC, Google Weave) to the [Device Abstraction Layer](doc:thnghub-dal-specification), allowing THNGHUB to communicate with the bridged devices and vice-versa.

The DAL is implemented as a message queue ([ZeroMQ](http://zeromq.org/)) using the Paranoid Pirate Protocol (https://rfc.zeromq.org/spec:6). As a consequence plugins can be written using [any language that supports ZeroMQ](http://zeromq.org/bindings:_start): from C to Node.js to Go.


### Mock Plugin

This plugin provides a REPL (Read-Eval-Print Loop) shell to interactively send commands to THNGHUB.

Start the plugin:

$ node plugins/mockPlugin/mockPlugin-repl.js

### Available Commands

The following commands are available when in the REPL shell.

* `changeId('XXXX')`
Changes the Thng ID (XXXX) that is used by the plugin. An initial value is set as one of the first variables in the script.

* `discoveryCreate()`
Sends a discovery-create DAL message for the currently active Thng ID.

* `discoveryDelete()`
Sends a discovery-delete DAL message for the currently active Thng ID.

* `propertyUpdate(n)`
Sends a properties DAL message for the currently active Thng ID with a key of `temp` and the passed value (n). If no value is passed, a random value is created.

* `createAction()`
Sends a Thng-action DAL message with a type of `scans` and a Thng for the currently active Thng ID.

* `multiplePropertyUpdates()`
Sends a properties DAL message 20 times with a key of `temp` and a random value.

* `propertyUpdatesForever(n)`
Sends a properties DAL message indefinitely with a key of `temp` and the passed value `n`, incrementing it by 5 each time the message is sent. If no value is passed, a random value is created.

* `cancelPropertyUpdatesForever()`
Stops the indefinite property update process (see above).
___

## Firmware updates

Firmware updates can be sent to devices connected to THNGHUB. This process is described below but please note the following:

* When the hub is started a `_firmwareUpdate` action type is created using the API key provided on startup
* The hub needs to be added to at least one collection

To apply a firmware update:

1. Create a `_firmwareUpdate` action on a collection the hub is part of (this allows an update to be sent to multiple hubs). This is the model for sending the action:

{
"type": "_firmwareUpdate",
"customFields": {
"firmwareUrl": "https://url/to/firmware",
"headers": {
"Content-Type": "text/plain"
},
"firmwareCheckSum": "checksumForFirmwareFile",
"endPointType": "typeOfEndpoint"
}
}

Note the following custom fields:

  * `firmwareUrl` - an HTTPS URL from which the firmware can be downloaded
  * `headers` - any HTTP headers that are required to download the firmware (could include 'Authorization' for example)
  * `firmwareCheckSum` - a SHA-2 checksum of the file
  * `endPointType` - the type of endpoint this firmware update should be applied to ('temperatureSensor' or 'waterSensor' for example)

2. When the hub receives the action it will attempt to download and store the file specified in the `customFields.firmwareUrl` field. Note that if the url is not https then the hub will not attempt the download.
3. The hub will then ensure the SHA-2 checksum of the file matches the value specified in `customFields.firmwareCheckSum`. If it does not then the firmware is deleted from the local filesystem and the plugin will not receive the action.
4. All plugins are informed new firmware is available as a 'collection-action' message (specified in Section 6.3 of the DAL). The `customFields.firmwareUrl` will be changed to point to the location of the file on the filesystem. For example: `file:///usr/local/firmware`.
5. It is then the plugin's responsibility to send the firmware to the devices it manages and monitor success/rollback.
6. The plugin should delete old firmware from the filesystem when new firmware is applied.
7. The plugin should update the properties corresponding to the firmware of each `endPointType` with the latest version number of the firmware that was applied.

Note that when a plugin registers with the hub it will receive any firmware updates already downloaded by the hub.

By default firmware is downloaded to a directory called `firmware` in the root of the THNGHUB project. The directory location can be changed by setting the `firmwareDownloadDirectory` configuration option (see 'Main configuration options' above).