Predictor Machine Learning

Enterprise Feature in beta

If you are interested in using this feature, please contact us.
Predictor™ is a beta capability only accessible to Enterprise users - we may make significant changes before the final version.

Predictor™: Machine Learning for Supply-chain Integrity

This tutorial will take you through the steps to create and deploy a machine learning model in your EVRYTHNG account. Predictor™ is the one stop shop in the EVRYTHNG platform for acquiring, training and deploying dedicated machine learning solutions. It offers a number of pre-packaged models for complex supply-chain integrity problems, which we developed with internal and external domain experts. Customers can select and install use case specific models in their account. These are not generic models. A model will be trained on the data available in the EVRYTHNG account. This means that the model is tailored to your processes and products.

A successfully trained model will transparently run predictions on actions whenever a user-generated action, for example when a user scans a product, is generated.

Steps to deploy a model

Predictor™ orchestrates the entire workflow, from dataset description to model training and finally to model deployment. The Predictor™ will also install the necessary applications and Reactor™ scripts in a user account. These scripts provide the glue to connect Predictor™ with the rest of the EVRYTHNG platform.

Selecting the model

GET https://ml.evrythng.io/machineLearning/models
Authorization:$OPERATOR_API_KEY

The process starts with selecting a type of model. List available models by accessing the /machineLearning/models endpoint.

POST https://ml.evrythng.io/machineLearning/models/scanningAnomalies
Authorization:$OPERATOR_API_KEY
{
    "projectId": "UrmDyx57gkE4PKfkYxkyNkwq"
}
{
    "accountId": "UFPhCBr2BgPRQKRawYHeDfHg",
    "createdAt": 1576242926846,
    "customFields": {
        "projectId": "UrmDyx57gkE4PKfkYxkyNkwq"
    },
    "id": "f3e2e6daac15444da3a5c34dd9ba8d15",
    "modelType": "scanningAnomalies",
    "resourceType": "models",
    "tags": [],
    "timestamp": 1576242926846
}

Each type of model is for a specific use case. For example, to detect abnormal product scans the model type "scanningAnomalies" can be used. When the model is created, it requires the project id. A model will be deployed to one project as a new EVRYTHNG application.

Download training data

POST https://ml.evrythng.io/machineLearning/models/scanningAnomalies/f3e2e6daac15444da3a5c34dd9ba8d15/datasets
Authorization:$OPERATOR_API_KEY
{
  "evrythngResources":["https://api.evrythng.com/actions/all?filter=timestamp>1576500118607&context=true"]
}
{
    "accountId": "UFPhCBr2BgPRQKRawYHeDfHg",
    "createdAt": 1576243116395,
    "customFields": {
        "evrythngResources": [
            "https://api.evrythng.com/actions/all?filter=timestamp>0&context=true"
        ]
   },
    "state": "running",
    "id": "9ed32c5c37fa469d814772baaa7cf486",
    "modelId": "f3e2e6daac15444da3a5c34dd9ba8d15",
    "modelType": "scanningAnomalies",
    "resourceType": "datasets",
    "tags": [],
    "timestamp": 1576243116395
}

Next, the raw training data must be downloaded from the EVRYTHNG account. A training set is described as one or more EVRYTHNG queries. For example, the query https://api.evrythng.com/actions/all?filter=timestamp>1576500118607&context=true returns all actions since 2019-12-16T13:41:43.895479.

Downloading will take a while. The dataset is ready when state is set to done. You can inspect the state of a resource anytime if you have its URL. For the dataset we created, the link is:

https://ml.evrythng.io/machineLearning/models/scanningAnomalies/f3e2e6daac15444da3a5c34dd9ba8d15/datasets/9ed32c5c37fa469d814772baaa7cf486

Transform the downloaded data

POST https://ml.evrythng.io/machineLearning/models/scanningAnomalies/f3e2e6daac15444da3a5c34dd9ba8d15/transformations
Authorization:$OPERATOR_API_KEY
{
 "datasetIds":["9ed32c5c37fa469d814772baaa7cf486"]
}
{
    "accountId": "UFPhCBr2BgPRQKRawYHeDfHg",
    "createdAt": 1576243528222,
    "customFields": {
        "datasetIds": [
            "9ed32c5c37fa469d814772baaa7cf486"
        ],
        "jobId": "transform_data_20191213_132528"
    },
    "state": "running",
    "id": "cf1acd76529a4218a82068be0a87e830",
    "modelId": "f3e2e6daac15444da3a5c34dd9ba8d15",
    "modelType": "scanningAnomalies",
    "resourceType": "transformations",
    "tags": [],
    "timestamp": 1576243528222
}

A model doesn't understand common data types such as strings or JSON objects. Therefore, the data must be transformed before it can be used to train a model, such as a deep neural network. The Predictor™ automatically determine which features are need and how to transform them.

Note that the dataset id 9ed32c5c37fa469d814772baaa7cf486 is the id of the previously created dataset.

GET https://ml.evrythng.io/machineLearning/models/scanningAnomalies/f3e2e6daac15444da3a5c34dd9ba8d15/transformations/cf1acd76529a4218a82068be0a87e830
Authorization:$OPERATOR_API_KEY
{
    "accountId": "UFPhCBr2BgPRQKRawYHeDfHg",
    "createdAt": 1576243528222,
    "customFields": {
        "bucketId": "f3e2e6daac15444da3a5c34dd9ba8d15",
        "datasetId": "9ed32c5c37fa469d814772baaa7cf486",
        "datasetIds": [
            "9ed32c5c37fa469d814772baaa7cf486"
        ],
        "jobId": "transform_data_20191213_132528",
        "modelType": "scanning_anomalies",
        "rootEvalDataOut": "gs://f3e2e6daac15444da3a5c34dd9ba8d15/9ed32c5c37fa469d814772baaa7cf486/cf1acd76529a4218a82068be0a87e830/working_dir/test_transformed",
        "rootTrainDataOut": "gs://f3e2e6daac15444da3a5c34dd9ba8d15/9ed32c5c37fa469d814772baaa7cf486/cf1acd76529a4218a82068be0a87e830/working_dir/train_transformed",
        "tftWorkingDir": "gs://f3e2e6daac15444da3a5c34dd9ba8d15/9ed32c5c37fa469d814772baaa7cf486/cf1acd76529a4218a82068be0a87e830/working_dir/scanning_anomalies_tft"
    },
    "state": "done",
    "id": "cf1acd76529a4218a82068be0a87e830",
    "modelId": "f3e2e6daac15444da3a5c34dd9ba8d15",
    "modelType": "scanningAnomalies",
    "resourceType": "transformations",
    "tags": 1576243528222,
    "timestamp": 1576243528222
}

Expect this step to take several minutes on a small dataset or up to an hour with a larger dataset. When the transformation is complete, (state is done), we're ready to train a model. When the state is done, proceed with training the model.

Train a model on the transformed training data

POST https://ml.evrythng.io/machineLearning/models/scanningAnomalies/f3e2e6daac15444da3a5c34dd9ba8d15/trainedModels
Authorization:$OPERATOR_API_KEY
{
 "tranformedDatasetIds":["cf1acd76529a4218a82068be0a87e830"]
}
{
    "accountId": "UFPhCBr2BgPRQKRawYHeDfHg",
    "createdAt": 1576246326511,
    "state": "running",
    "id": "c0d01c5e8e544148a2ca4fcf9126040f",
    "modelId": "f3e2e6daac15444da3a5c34dd9ba8d15",
    "modelType": "scanningAnomalies",
    "resourceType": "trainedModels",
    "tags": [],
    "timestamp": 1576246326511
}

The model is trained on the data we transformed in the previous step.

Deploy model to your account

POST https://ml.evrythng.io/machineLearning/models/scanningAnomalies/f3e2e6daac15444da3a5c34dd9ba8d15/deployments
Authorization:$OPERATOR_API_KEY
{
 "trainedModelIds":["c0d01c5e8e544148a2ca4fcf9126040f"]
}
{
    "accountId": "UFPhCBr2BgPRQKRawYHeDfHg",
    "createdAt": 1576247298171,
    "customFields": {
        "createdAt": 1576247312114,
        "metadata": {},
        "trainedModelIds": [
            "c0d01c5e8e544148a2ca4fcf9126040f"
        ],
        "type": "simple",
        "updatedAt": 1576247312114
    },
    "id": "0a48d97a93b542ee939ba6168a482465",
    "modelId": "f3e2e6daac15444da3a5c34dd9ba8d15",
    "modelType": "scanningAnomalies",
    "resourceType": "deployments",
    "tags": [],
    "timestamp": 1576247298171,
    "state": "done"
}

A trained model needs to be deployed to your account. To do this, we'll create a new deployments endpoint for the trained model.

Testing your model

Return to the dashboard. If the model was deployed successfully, you will see a new app in the project used to create the scanning anomalies model instance. Try scanning a THNG's QR code a couple of time. You should start to see new actions called _mlpredictions in your dashboard.

Updated about a month ago

Predictor Machine Learning


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.