Identifier Recognition

EVRYTHNG offers an Identifier Recognition service through the REST API. Developers can use this resource to identify barcodes in images, and any linked products or Thngs linked to those codes. A common use of this capability is when a web app is used by consumers to scan a QR code on the packaging of their purchase in order to learn more information about the product, such as provenance, traceability history, access rewards, and so on.

Any Application and Operator API key can POST an image, and if a product/Thng redirection or barcode matches the image, the service returns information about the matches made. This includes any redirection information set up for the resource that matched the image recognition query. In the case that a barcode is found but it does not link to a Thng or product in scope of the authenticating API key, the raw decoded data is returned in case it can still be of use.

📘

Walkthrough

For a complete walkthrough of the process of utilizing product recognition, read Real-time Web Product Scanning.

API Status
General Availability:
/scan/identifications
tn.gg/redirections/:shortId/ir
Beta:
/scan/identifications when method=ocr
/scan/identifications when method=digimarc

Enabling Recognition

To enable recognition of a specific type of barcode, include the appropriate product identifier as an item in the identifiers field in the Thng or product to be recognised when scanned. If a scanned image contains the barcode listed in these identifiers, that resource is returned.

This can be done through the REST API, or via the Dashboard. The value of the key should be the numerical value encoded within the barcode itself as a string. For example a barcode containing 96385074 as an EAN 8 barcode will match the following Thng:

{
  "name": "Item #68957",
  "description": "An item with a barcode identifier to aid in barcode recognition",
  "tags": ["object", "scan-friendly", "barcode"],
  "identifiers: {
    "ean_8": "96385074"
  }
}

Alternatively, if the scanned image is a Thng or product redirection QR code, you can also use the encoded URL as the value field, where applicable. QR codes generated when adding a redirection for a given Thng or product will also be matched independently of the identifiers listed through their redirection's shortId.

Performing a Scan or Identification

Scanning an image is performed by making a POST request to /scan/identifications through the API, or using the scan() method in the scanthng.js SDK. The filter query parameter specified dictates which types of codes are searched for or recognition methods are attempted on the supplied image payload.

Similarly, a GET request to the same resource will attempt to find any products or Thngs visible to the authenticating Application or Operator API Key with a code containing the value query parameter value. For example, a QR code with tn.gg redirection to a product would match a value=https://tn.gg/yyZZ1nAh query parameter value.

See Identify from an Image or Read an Identifier's Product or Thng for example requests.

Filter Format

The filter query parameter can contain the following items:

• method - String
  The scanning method to use.

• type - String
  The type of identifier to identify.

You can optionally also include the following parameters in the URL query string:

• perPage - Integer
  Specify how many items to return per page of results.

• debug - Boolean
  Set to true to view extra information used to measure performance.

🚧

Important

The filter text must be URL encoded, similar to other Platform filters.

The different types of methods and identifier types are shown in the table below:

Matching and Response

When a code type matching the filter specified is found in the image, the Identifier Recognition API looks for a Thng/product that is visible to the API key that performed the call (i.e.: in the same scope).

The response is an array containing metadata and a set of matching results, if any were found. If there were no results, or an incompatible combination of method and type were specified, an empty array will be returned. If the scan was performed on behalf of a user, their object will also be included, allowing extra functionality such as Scan actions. An example is shown below:

HTTP/1.1 200 OK
Content-Type: application/json

[
  {
    "results": [
      {
        "redirections": [
          "https://tn.gg/Q9Wqcg4w"
        ],
        "product": {
          "id": "UYKNDMGcswyFBdf6wr7M5Erm",
          "properties": {},
          "name": "Box of Cereals",
          "identifiers": {}
        }
      }
    ],
    "meta": {
      "method": "2d",
      "score": 100,
      "value": "https://tn.gg/Q9Wqcg4w",
      "type": "qr_code"
    }
  }
]

Data Fields

• results - Object array
  Array of identification results, containing the matching Thng/product and any redirections associated with it.

• meta - Object
  Object containing metadata about the identification operation.

  • meta.method - String
    The method used to identify the product/Thng.

  • meta.score - Number
    Confidence score of the results for the ir method, or 100 for one or more matches for the other method values, else 0 if no Thngs or products were found.

  • meta.value - String
    The actual value encoded within the identifier. Can be used with a GET request to retrieve the associated Thng/product.

  • meta.type - String
    The type used to identify the Thng/product.

Payload Sources

The /scan/identifications endpoint accepts JPG and PNG images as a file upload or Base64-encoded image data as a string.

File Upload

You can upload a file either from a file browser or a direct camera capture on mobile devices. The scanthng.js library includes this functionality as part of its API. When the scan() method is invoked, the device-specific native dialog for choosing a file or capturing from a camera is shown to the user. See below for examples.

Base64 Data

As an alternative to uploading a file through evrythng.js, Base64 image data may also be submitted in the request body. The image must be encoded in a string containing the MIME type header before the actual image data (i.e.: "data:{content-type},...{Base64 data}...") and be nested inside a JSON object with the image key. A truncated example is shown below:

{
  "image": "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAAS..."
}

Example File Uploads

File Upload

evrythng.use(ScanThng);

const app = new evrythng.Application(APPLICATION_API_KEY);

// Launch image capture or file upload for image to scan
// The 'filter' chosen here will look for a 2D QR code
app.scan({
  filter: { method: '2d', type: 'qr_code' },
}).then(console.log)
  .catch(alert);

Base64 Data

POST /scan/identifications?filter=method%3D<method>%26type%3D<type>
Content-Type: application/json
Authorization: $APPLICATION_API_KEY

{
  "image": Base64 Data
}

curl -i -H "Content-Type: application/json" \
  -H "Authorization: $APPLICATION_API_KEY" \
  -X POST 'https://api.evrythng.com/scan/identifications?filter=method%3D<method>%26type%3D<type>' \
  -d '{
    "image": "data:image/jpeg;base64,/9j/4AAQSkZJRgABAQAASABIAAD/4QCARAAKg..."
  }'

evrythng.use(ScanThng);

const app = new evrythng.Application(APPLICATION_API_KEY);

const data = 'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAA...';
const options = { filter: 'method=2d&type=qr_code' };

app.scan(data)
  .then(console.log)
  .catch(alert);

Identify from an Image

🚧

Important

The filter text must be URL encoded, similar to other Platform filters.

When a request is made to the /scan/identifications endpoint with the POST method, the image is analysed according to the filter query string provided, and one or more matches to products/Thngs are produced if they list the matching identifiers values.

POST /scan/identifications?filter=method=<method>&type=<type>
Content-Type: application/json
Authorization: $APPLICATION_API_KEY

{
  "image": Base64 Data
}

# The `image` value below has been truncated for brevity.

curl -H "Content-Type: application/json" \
  -H "Authorization: $APPLICATION_API_KEY" \
  -X POST 'https://$EVT_API_DOMAIN/scan/identifications?filter=method%3D2d%26type%3Dqr_code' \
  -d '{
    "image": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAIAAAACACAYAA..."
  }'

const filter = { method: '2d', type: 'qr_code' };

app.scan({ filter }).then(console.log)
  .catch(alert);

const request = require('request');

const APPLICATION_API_KEY = 'xnDkdhezgJMnx6fGr4xMAxzPJpgM3xc0sbf...';

const FILTER = 'method=2d&type=qr_code';
const BASE_64 = 'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAgAA...';

request({
  url: `https://api.evrythng.com/scan/identifications?filter=${FILTER}`,
  method: 'POST',
  headers: { Authorization: APPLICATION_API_KEY },
  json: { image: BASE_64 }
}, (err, response, body) => {
  if(err) {
    console.log(err);
    return;
  }

  console.log(JSON.stringify(body, null, 2));
});

Read an Identifier's Product or Thng

If a value has already been extracted from an image with a decoding library, or via the POST /scan/identifications endpoint, you can read the associated product or Thng with a GET request containing a filter specifying the type of the identifier.

📘

Note

The type parameter must be specified with the submitted value for the identification to succeed. For example, type: 'qr_code' when identifying a URL value: 'https://tn.gg/yyZZ1nAh' from a QR code.

GET /scan/identifications?filter=type=<type>&value=<value>
Authorization: $APPLICATION_API_KEY

curl -H "Authorization: $APPLICATION_API_KEY" \
  -X GET 'https://$EVT_API_DOMAIN/scan/identifications?filter=type%3Dqr_code%26value%3Dhttps://tn.gg/yyZZ1nAh'

// Get identification info for pre-obtained value 'https://tn.gg/yyZZ1nAh'
const filter = { type: 'qr_code', value: 'https://tn.gg/yyZZ1nAh' };

app.identify({ filter }).then(console.log)
  .catch(alert);

HTTP/1.1 200 OK
Content-Type: application/json

[
  {
    "results": [
      {
        "redirections": [
          "https://tn.gg/yyZZ1nAh"
        ],
        "product": {
          "id": "UYKNDMGcswyFBdf6wr7M5Erm",
          "properties": {},
          "name": "Box of Cereals",
          "identifiers": {}
        }
      }
    ],
    "meta": {
      "method": "2d",
      "score": 100,
      "value": "https://tn.gg/yyZZ1nAh",
      "type": "qr_code"
    }
  }
]

Create an IR Reference Image

Upload a local file as a reference image for a given redirection short ID.

POST https://tn.gg/redirections/:shortId/ir
Content-Type: image/jpeg
Authorization: $OPERATOR_API_KEY

File

curl -H "Authorization: $OPERATOR_API_KEY" \
  'https://tn.gg/redirections/G53eYMM3/ir' \
  -F '[email protected]'

HTTP/1.1 201 Created
Content-Type: application/json

{
  "id": "5ab9159fa898950028b54a55",
  "name": "5ab90ecbaedf0700266aa412_b98e27f0-310c-11e8-8461-db50a78858ff",
  "thumb60": "https://prod-scan-app-bucket-1x12op27qs6xd.s3.amazonaws.com/upload_cfc010020ae357f0f9dc0e64334bcf57_60.jpg",
  "thumb120": "https://prod-scan-app-bucket-1x12op27qs6xd.s3.amazonaws.com/upload_cfc010020ae357f0f9dc0e64334bcf57_120.jpg",
  "file": "https://prod-scan-app-bucket-1x12op27qs6xd.s3.amazonaws.com/upload_cfc010020ae357f0f9dc0e64334bcf57.jpg"
}

Read all IR Reference Images

Read all uploaded reference images for a given short ID.

GET https://tn.gg/redirections/:shortId/ir
Authorization: $OPERATOR_API_KEY

curl -H "Authorization: $OPERATOR_API_KEY" \
  -X GET https://tn.gg/redirections/G53eYMM3/ir

HTTP/1.1 200 OK

[
  {
    "id": "5ab90eccaedf0700266aa413",
    "name": "5ab90ecbaedf0700266aa412_a7c735b0-3108-11e8-b46e-c7973462d6ac",
    "thumb60": "https://prod-scan-app-bucket-1x12op27qs6xd.s3.amazonaws.com/upload_97d0df0d6e0a67d0b27cc8affce9dc40_60.jpg",
    "thumb120": "https://prod-scan-app-bucket-1x12op27qs6xd.s3.amazonaws.com/upload_97d0df0d6e0a67d0b27cc8affce9dc40_120.jpg",
    "file": "https://prod-scan-app-bucket-1x12op27qs6xd.s3.amazonaws.com/upload_97d0df0d6e0a67d0b27cc8affce9dc40.jpg"
  },
  {
    "id": "5ab9141d612c11003155af7d",
    "name": "5ab90ecbaedf0700266aa412_d31ffb90-310b-11e8-84a2-852d87264559",
    "thumb60": "https://prod-scan-app-bucket-1x12op27qs6xd.s3.amazonaws.com/upload_0f996323948b67b8c07d7e172f795ad9_60.jpg",
    "thumb120": "https://prod-scan-app-bucket-1x12op27qs6xd.s3.amazonaws.com/upload_0f996323948b67b8c07d7e172f795ad9_120.jpg",
    "file": "https://prod-scan-app-bucket-1x12op27qs6xd.s3.amazonaws.com/upload_0f996323948b67b8c07d7e172f795ad9.jpg"
  },
  {
    "id": "5ab9148d98d2f700221d7629",
    "name": "5ab90ecbaedf0700266aa412_15eaad30-310c-11e8-a6fe-2d2710455ba9",
    "thumb60": "https://prod-scan-app-bucket-1x12op27qs6xd.s3.amazonaws.com/upload_9cd4eecf06a1af38aa39bd3f11ad35e6_60.jpg",
    "thumb120": "https://prod-scan-app-bucket-1x12op27qs6xd.s3.amazonaws.com/upload_9cd4eecf06a1af38aa39bd3f11ad35e6_120.jpg",
    "file": "https://prod-scan-app-bucket-1x12op27qs6xd.s3.amazonaws.com/upload_9cd4eecf06a1af38aa39bd3f11ad35e6.jpg"
  }
]

Delete an IR Reference Image

Remove a previously uploaded reference image by id.

DELETE https://tn.gg/redirections/:shortId/ir/:id
Authorization: $OPERATOR_API_KEY

curl -H "Authorization: $OPERATOR_API_KEY" \
  -X DELETE https://tn.gg/redirections/G53eYMM3/ir/5ab917fea898950028b54a56

HTTP/1.1 200 OK

Scan Actions

A scans action created upon a successful recognition generates valuable contextual data which can be used for business rules/decision making and data insights analysis. For example: geolocation, browser and device type, anonymous user ID.

Creating a scans action can done using the result returned by scan() with the evrythng.js SDK. An example is shown below:

📘

Note

Since an action cannot be created with an Application API Key, the createAnonymousUser: true option is required in order to be able to create a Scan action as an Application User. Alternatively, a regular Application User API Key can also be used.

app.scan({
  filter: { method: '2d', type: 'dm' },
  createAnonymousUser: true,
})
  .then((res) => {
    // Create a Scan action as the anonymous user for the first result
    const result = res[0].results[0];  
    return result.thng.action('scans').create();
  })
  .catch(alert);

With this option enabled, you will receive a lot more useful data items, shown in the example below. Most notable are the addition of the location, and createdBy... items, which can be used for analysis and decision making when aggregated across a large number of scans.

{
  "id": "UkAsQqTWN2y2tfwVkyBKykws",
  "createdAt": 1480522789344,
  "timestamp": 1480522789344,
  "type": "scans",
  "user": "UFAsQqTRtEf5FpRekYC5CgKg",
  "location": {
    "latitude": 51.5142,
    "longitude": -0.0931,
    "position": {
      "type": "Point",
      "coordinates": [
        -0.0882956,
        51.5303183
      ]
    }
  },
  "locationSource": "geoIp",
  "createdByProject": "U2As9WsMthW5F5aeFECpCeDs",
  "createdByApp": "UFUstWQVQEC5FpRe2EWpWBhf",
  "product": "UFd8t2aaN2y2tCwBFTepTgEs"
}

Redirections

Once an identifier has been recognized, the results can be used to redirect the user to a pre-set destination according to any redirections attached to the product/Thng associated with the result. For example:

// Perform a scan for identifiers
app.scan({
  filter: { method: '2d', type: 'qr_code' },
  createAnonymousUser: true,
})
  .then((res) => {
    // Redirect to the first redirection set on the resource
    const result = res[0].results[0];  
    return app.redirect(result.redirections[0]);
  })
  .catch(alert);

Error Codes

The table below details the error codes that may be returned by the Identifier Recognition API:

Known Issues

• Denying access to location in Firefox, IE11, and Edge browsers can cause issues with scanthng.js. Make sure to guide users to always accept the request for location data.

• Using a web app or page in Safari Incognito mode can cause issues with scanthng.js. Apps should display a message instructing users to not use Safari Incognito mode.