Predictor Machine Learning
Enterprise Feature in beta
If you're interested in using this feature, contact us.
Predictorâ„¢ is a beta capability accessible only to Enterprise users. We might make significant changes before the final version is released.
Predictorâ„¢: Machine Learning for Supply-chain Integrity
This tutorial takes you through the steps to create and deploy a machine learning model in your EVRYTHNG account. Predictorâ„¢ is the one-stop shop in the EVRYTHNG platform for acquiring, training, and deploying dedicated machine learning solutions. It offers a number of pre-packaged models for complex supply-chain integrity problems, which we developed with internal and external domain experts. Customers can select and install use case specific models in their accounts. These aren't generic models; a model is trained on the data available in the EVRYTHNG account. This means that the model is tailored to your processes and products.
A successfully trained model transparently runs predictions on actions whenever a user-generated action, for example when someone scans a product, is generated.
Steps to deploy a model
Predictorâ„¢ orchestrates the entire workflow, from dataset description to model training and finally to model deployment. The Predictorâ„¢ also installs the necessary applications and Reactorâ„¢ scripts in a user account. These scripts provide the glue to connect Predictorâ„¢ with the rest of the EVRYTHNG platform.
Selecting the model
GET https://ml.evrythng.io/machineLearning/models
Authorization:$OPERATOR_API_KEY
The process starts with selecting a type of model. List available models by accessing the /machineLearning/models endpoint.
POST https://ml.evrythng.io/machineLearning/models/scanningAnomalies
Authorization:$OPERATOR_API_KEY
{
"projectId": "UrmDyx57gkE4PKfkYxkyNkwq"
}
{
"accountId": "UFPhCBr2BgPRQKRawYHeDfHg",
"createdAt": 1576242926846,
"customFields": {
"projectId": "UrmDyx57gkE4PKfkYxkyNkwq"
},
"id": "f3e2e6daac15444da3a5c34dd9ba8d15",
"modelType": "scanningAnomalies",
"resourceType": "models",
"tags": [],
"timestamp": 1576242926846
}
Each type of model is for a specific use case. For example, to detect abnormal product scans the model type "scanningAnomalies" can be used. When the model is created, it requires the project id. A model is deployed to one project as a new EVRYTHNG application.
Download training data
POST https://ml.evrythng.io/machineLearning/models/scanningAnomalies/f3e2e6daac15444da3a5c34dd9ba8d15/datasets
Authorization:$OPERATOR_API_KEY
{
"evrythngResources":["https://api.evrythng.com/actions/all?filter=timestamp>1576500118607&context=true"]
}
{
"accountId": "UFPhCBr2BgPRQKRawYHeDfHg",
"createdAt": 1576243116395,
"customFields": {
"evrythngResources": [
"https://api.evrythng.io/v2/actions/all?filter=timestamp>0&context=true"
]
},
"state": "running",
"id": "9ed32c5c37fa469d814772baaa7cf486",
"modelId": "f3e2e6daac15444da3a5c34dd9ba8d15",
"modelType": "scanningAnomalies",
"resourceType": "datasets",
"tags": [],
"timestamp": 1576243116395
}
Next, the raw training data must be downloaded from the EVRYTHNG account. A training set is described as one or more EVRYTHNG queries. For example, the query https://api.evrythng.com/actions/all?filter=timestamp>1576500118607&context=true returns all actions since 2019-12-16T13:41:43.895479.
Downloading takes a while. The dataset is ready when state is set to done. You can inspect the state of a resource anytime if you have its URL. For the dataset in the example, the link is:
https://ml.evrythng.io/machineLearning/models/scanningAnomalies/f3e2e6daac15444da3a5c34dd9ba8d15/datasets/9ed32c5c37fa469d814772baaa7cf486
Transform the downloaded data
POST https://ml.evrythng.io/machineLearning/models/scanningAnomalies/f3e2e6daac15444da3a5c34dd9ba8d15/transformations
Authorization:$OPERATOR_API_KEY
{
"datasetIds":["9ed32c5c37fa469d814772baaa7cf486"]
}
{
"accountId": "UFPhCBr2BgPRQKRawYHeDfHg",
"createdAt": 1576243528222,
"customFields": {
"datasetIds": [
"9ed32c5c37fa469d814772baaa7cf486"
],
"jobId": "transform_data_20191213_132528"
},
"state": "running",
"id": "cf1acd76529a4218a82068be0a87e830",
"modelId": "f3e2e6daac15444da3a5c34dd9ba8d15",
"modelType": "scanningAnomalies",
"resourceType": "transformations",
"tags": [],
"timestamp": 1576243528222
}
A model doesn't understand common data types such as strings or JSON objects. Therefore, the data must be transformed before it can be used to train a model, such as a deep neural network. The Predictorâ„¢ automatically determines which features are needed and how to transform them.
Note that the dataset id 9ed32c5c37fa469d814772baaa7cf486 is the id of the previously created dataset.
GET https://ml.evrythng.io/machineLearning/models/scanningAnomalies/f3e2e6daac15444da3a5c34dd9ba8d15/transformations/cf1acd76529a4218a82068be0a87e830
Authorization:$OPERATOR_API_KEY
{
"accountId": "UFPhCBr2BgPRQKRawYHeDfHg",
"createdAt": 1576243528222,
"customFields": {
"bucketId": "f3e2e6daac15444da3a5c34dd9ba8d15",
"datasetId": "9ed32c5c37fa469d814772baaa7cf486",
"datasetIds": [
"9ed32c5c37fa469d814772baaa7cf486"
],
"jobId": "transform_data_20191213_132528",
"modelType": "scanning_anomalies",
"rootEvalDataOut": "gs://f3e2e6daac15444da3a5c34dd9ba8d15/9ed32c5c37fa469d814772baaa7cf486/cf1acd76529a4218a82068be0a87e830/working_dir/test_transformed",
"rootTrainDataOut": "gs://f3e2e6daac15444da3a5c34dd9ba8d15/9ed32c5c37fa469d814772baaa7cf486/cf1acd76529a4218a82068be0a87e830/working_dir/train_transformed",
"tftWorkingDir": "gs://f3e2e6daac15444da3a5c34dd9ba8d15/9ed32c5c37fa469d814772baaa7cf486/cf1acd76529a4218a82068be0a87e830/working_dir/scanning_anomalies_tft"
},
"state": "done",
"id": "cf1acd76529a4218a82068be0a87e830",
"modelId": "f3e2e6daac15444da3a5c34dd9ba8d15",
"modelType": "scanningAnomalies",
"resourceType": "transformations",
"tags": 1576243528222,
"timestamp": 1576243528222
}
Expect this step to take several minutes on a small dataset or up to an hour with a larger dataset. When the transformation is complete (state is done), you're ready to train a model. When the state is done, proceed with training the model.
Train a model on the transformed training data
POST https://ml.evrythng.io/machineLearning/models/scanningAnomalies/f3e2e6daac15444da3a5c34dd9ba8d15/trainedModels
Authorization:$OPERATOR_API_KEY
{
"tranformedDatasetIds":["cf1acd76529a4218a82068be0a87e830"]
}
{
"accountId": "UFPhCBr2BgPRQKRawYHeDfHg",
"createdAt": 1576246326511,
"state": "running",
"id": "c0d01c5e8e544148a2ca4fcf9126040f",
"modelId": "f3e2e6daac15444da3a5c34dd9ba8d15",
"modelType": "scanningAnomalies",
"resourceType": "trainedModels",
"tags": [],
"timestamp": 1576246326511
}
The model is trained on the data we transformed in the previous step.
Deploy the model to your account
POST https://ml.evrythng.io/machineLearning/models/scanningAnomalies/f3e2e6daac15444da3a5c34dd9ba8d15/deployments
Authorization:$OPERATOR_API_KEY
{
"trainedModelIds":["c0d01c5e8e544148a2ca4fcf9126040f"]
}
{
"accountId": "UFPhCBr2BgPRQKRawYHeDfHg",
"createdAt": 1576247298171,
"customFields": {
"createdAt": 1576247312114,
"metadata": {},
"trainedModelIds": [
"c0d01c5e8e544148a2ca4fcf9126040f"
],
"type": "simple",
"updatedAt": 1576247312114
},
"id": "0a48d97a93b542ee939ba6168a482465",
"modelId": "f3e2e6daac15444da3a5c34dd9ba8d15",
"modelType": "scanningAnomalies",
"resourceType": "deployments",
"tags": [],
"timestamp": 1576247298171,
"state": "done"
}
A trained model needs to be deployed to your account. To do this, create a deployments endpoint for the trained model.
Testing your model
Return to the dashboard. If the model was deployed successfully, a new app is shown in the project used to create the scanning anomalies model instance. Try scanning a Thng's QR code a couple of times. You'll start to see new actions called _mlpredictions on your dashboard.
Updated almost 2 years ago