Skip to main content

Decoder Intro

Decoder Systems​

Once again, KORA delivers innovation through a new and robust system for processing data received on the platform. Advanced users are no longer limited to existing system defaults and can create tailor-made functions for special data processing and updating internal variables.

The amazing thing is that each user, within their client account, can create their own function library to use anytime.

Libraries​

In order to access the system, click on the new item “Libraries”, which is available on the platform menu.

In case this feature is not available for your user, please reach out to KORE’s Technical Support.

Libraries - Step 1

This screen shows the list of all your available functions. In the beginning, there are no implemented functions in your library for sure, but do not worry, as soon you will have lots of them.

The system allows functions to be filed in folders and subfolders for better organizing your library.

To start, click on button “+New folder”, within “Decoders”, on the left-hand side menu. An additional window will be displayed for you to name the new folder. After naming the folder, click on the “Create folder” button.

Libraries - Step 2

From this point, you may want to create subfolders within your organizational structure, or directly create a new decoder. To simplify, we will save the decoder directly in your main folder previously created, by clicking on “+ New decoder”.

Libraries - Step 3

This will display, on the left-hand side, a screen for decoder function creation/edition and a basic template of the sourcing code as an example.

Insert a name for the new decoder and push the button “SAVE” on the right-hand side of the screen .

Function Editor​

Congratulations to our team of developers at KORE! They have devised a robust IDE for creating and editing functions in JavaScript. Let’s see how it works:

Function Details​

On the top of the page, you can find the fields for identifying the function within your library.

  • Name: use it for naming the function. Try to briefly describe what it does, preferably using a reference of the data type received, processing or even the specific device with which it operates. Attention: always begin with the name. Functions without name cannot be saved.
  • Tags: Those who are familiar with the platform already know the TAG concept. Those are keywords (tags or labels) to help in the search and filtering. For example, you may use as TAG the type or device name for which the function was developed. Do not forget to click on the arrow at the right-hand side of the field so that the typed Tag may be saved in the list.

JS Editor​

On the central left-hand side of the screen, we can find the code editor. This is a standard editor, with automatic interpreter of everything that is being typed, which shows the automatic completion options by the language pattern.

We can see the line numbering and a thumbnail on the right-hand side, next to the scroll bar, which represents how much the total code is being shown on the screen at that moment.

Whenever there is a typing error in the code, this thumbnail will show a red line corresponding to the location of the problem found.

The editor has an automatic recording system to save changes and, on the left-hand bottom side you can see the recording confirmation (“SAVED”). If you want to make sure changes are recorded, you can click on the “SAVE” button, on the top right-hand side of the page.

Json Editor - Step 1

Test Payload​

There is an area on the right-hand top side where we need to insert the payload that will be used during tests to run the function. This payload must be the complete package received by the platform, on JSON format. If it is a LoRaWAN device, for example, it is the package with all data received from the Network Server (LNS), besides the payload of data sent by your device.

After you enter a valid payload, always confirm if the function is saved (otherwise, click on the “SAVE” button).

Once a correct payload test has been entered on the correct location, you can click on the button to run the function (blue arrow on the upper right corner). Now the magic happens. The system will simulate the actual execution of the function, by considering the JSON of the field “Test Payload” as input. Outcome will be displayed on next window.

Test Payload - Step 1

Server Response​

This window will display, in descending chronological order, all results of tests run.

If everything turns out well, a “SUCCESS” message will be displayed and it will be possible to analyze the return of your function, by clicking on the expansion arrow.

Server Response - Step 1

If the function is not correct, the corresponding “ERROR” message will be displayed.

Server Response - Step 2

If you wish to clear the result list, just click on the recycling blue button.

Creating your first function​

As a first example, let’s create an elementary function that can receive a list of variables and generate an output in the required pattern. We will format the output with the Kora accepted pattern, but any other pattern can be used in a similar way.

However, we first need to understand what the input and output patterns of a customized function are.

Input​

As an input pattern, the system will automatically send the JSON to the function, with the full payload received by the platform. For example, let’s look at a payload pattern received by a Network Server, in the LoRAWAN protocol:

{
  "timestamp": 1597092154235,
  "meta": {
    "network": "a653ee9da56a43b9ab308c3b2f06eb81",
    "gateway": "000080029c4572cd",
    "device": "70b3d54b1c1146b3",
    "device_addr": "17566514",
    "history": true,
    "application": "70b3d54b1cffff9e",
    "packet_id": "5735e2038b3f1d82ca7c66046f2fdfb0",
    "packet_hash": "0f64a9c62540a9abf8622da0bd2c2596",
    "time": 1597065851.947938
  },
  "params": {
    "rx_time": 1597065851.9132946,
    "counter_up": 181,
    "port": 1,
    "encrypted_payload": "46bM/bXY1swUJAINhlAli6z7KQ==",
    "payload": "GgIbAAQQD1YGBCQhAAEFBQkHxA==",
    "radio": {
      "time": 1597065851.9132946,
      "freq": 916,
      "modulation": {
        "type": "LORA",
        "bandwidth": 125000,
        "spreading": 9,
        "coderate": "4/5"
      },
      "hardware": {
        "chain": 1,
        "channel": 4,
        "tmst": 961562868,
        "status": 1,
        "rssi": -64,
        "snr": 12.3
      },
      "datarate": 3,
      "delay": 0.009529352188110352,
      "size": 32
    },
    "lora": {
      "mac_commands": [],
      "header": {
        "ack": false,
        "adr": false,
        "confirmed": false,
        "type": 2,
        "version": 0,
        "adr_ack_req": false
      }
    },
    "duplicate": false
  },
  "type": "uplink"
}

Please note that we are talking about the full payload and not just the list of data sent by the device which, in that case, would be inside the JSON element.

params.payload: "GgIbAAQQD1YGBCQhAAEFBQkHxA==”.

The JSON will always be associated to the internal variable called payload - which is JSON type object - so that we can handle this JSON within our function.

Besides the “params.payload” field, there is also the “params.encrypted_payload” field, which is related to the device data field before it is decrypted. The important thing to know is that the “params.payload” field only will exist if it is stated on the device configuration that the “Security Encryption” is performed on the Network Service (NS). In order to check or fix it, enter the device details:

Function Input - Step 1

Return​

As to the customized function return, we need to return a JSON object with the following pattern, when using via Kora platform:

// {
//   "speed": { //Theobjectkeywillbethealiasofyourvariable
//     "name": "speed", // Variable name
//     "value": 35, // Variable value
//     "unit": "mhp" // variable unit (optional)
//   }
// }

The JSON elements must be repeated for all the return desired variables.

The object must always be stored on the response variable, which will, in fact, be the output JSON sent by KORA forward.

Basic Body​

This way, we can assume we need a basic body for our function to work within the system:

/* **************************************************************************
    // Main (basic default)
    // converts the received payload into Kora standard
    response = convertKora(payload);
************************************************************************** */

We use the convertKora(payload) user function (you can choose any name you want) , which will be responsible for interpreting and converting variables. The return of this function will be associated with the response variable. The response variable will internally be used by the platform for updating the variables for each specific device.

Conversion Function​

For the present example, suppose we are getting an even simpler input JSON than the LoRaWAN model, but that needs to be adapted to the right output pattern:

[
  {},
  {
    "name": "rpm",
    "value": 2190
  },
  {
    "name": "fuel",
    "value": 97,
    "unit": "%"
  },
  {
    "name": "location_car",
    "value": "car",
    "location": {
      "lat": -45.12356,
      "lng": -44.95116
    }
  },
  {
    "value": 2
  },
  {
    "name": "brake",
    "value": false
  }
]

In this very simple case, we have a direct list of several data which will only need to be converted to the right pattern. This will be done through the aforementioned convertKora function.

function convertKora(payload) {
  // The function reduce() will be used to perform a treatment function // of the data for each element of the received JSON
  const newPayload = payload.reduce(function (acc, pay) {
    let obj;
    switch (pay.name) {
      // Special treatment for location (lat and long)
      case "location_car":
        obj = {
          ...acc,
          ["location_car_lat"]: {
            name: "location_car_lat",
            value: pay.location.lat,
          },
          ["location_car_lngt"]: {
            name: "location_car_lng",
            value: pay.location.lng,
          },
        };
        break;
      // Special treatment for boolean field
      case "brake":
        obj = {
          ...acc,
          [pay.name]: {
            name: pay.name,
            value: pay.value ? 1 : 0,
          },
        };
        break;
      // Default treatment
      default:
        obj = {
          // the accumulator (acc) is used to keep the list already built and // introduce a new element (variable) at each pass
          ...acc,
          // The key is the varible alias
          [pay.name]: {
            // variable name name: pay.name, // variable value value: pay.value, // variable unit unit: pay.unit
          },
        };
    }
    return obj;
  }, {});
  // returns the new JSON object converted
  return newPayload;
}

Now, let’s interpret how this function works.

The main method used here is the payload.reduce() , which will pick the JSON object received (payload) and perform a specific function for each element inside it. Please note we u sed an accumulator (acc) to store the result after processing each element (pay). This way, we will have a final object with a list of all processed variables, which will ultimately be the return to our main function.

Last but not least, is the switch(pay.name). This is required because on the input JSON we have different elements and some of them have more than one piece of information, which may generate more than one output variable to follow the Kora pattern. Take the location_car case, which may generate different variables for latitude and longitude.

// Special treatment for location (lat and long) case 'location_car':
obj = {
    ...acc, ["location_car_lat"]: {
        name: "location_car_lat",
        value: pay.location.lat,
    },
    ["location_car_lngt"]: { name: "location_car_lng", value: pay.location.lng }
};
break;

The brake element is also treated differently, as it has two Boolean-type (false or true) values and the output must be of numeric type (0 or 1).

// Special treatment for boolean field case 'brake':
obj={
    ...acc, [pay.name]: {
        name: pay.name,
        value: pay.value ? 1 : 0
    }
};
break;

For all other elements, treatment is default.

// Default treatment default:
obj = {
  // the accumulator (acc) is used to keep the list already built and // introduce a new element (variable) at each pass
  ...acc,
  // The key is the varible alias
  [pay.name]: {
    // variable name name: pay.name, // variable value value: pay.value, // variable unit unit: pay.unit
  },
};

Full function (comment)​

// ********************************************************************** // Every payload decoding routine has 2 parameters
// @param {String} payload - Payload sent by device
// @param {Object} response - Decoded payload for response
// **********************************************************************
// *************************************************************************** // Example function to transform original payload to Kora format
// Receives a JSON object with the full payload originally received
// The new converted JSON object will be returned by the function
// The response format must be:
// {
//   "speed": { //Theobjectkeywillbethealiasofyourvariable
//     "name": "speed", // Variable name
//     "value": 35, // Variable value
//     "unit": "mhp" // variable unit (optional)
//   }
// }
// ************************************************************************

function convertKora(payload) {
  // The function reduce() will be used to perform a treatment function // of the data for each element of the received JSON
  const newPayload = payload.reduce(function (acc, pay) {
    var obj;
    switch (pay.name) {
      // Special treatment for location (lat and long)
      case "location_car":
        obj = {
          ...acc,
          ["location_car_lat"]: {
            name: "location_car_lat",
            value: pay.location.lat,
          },
          ["location_car_lngt"]: {
            name: "location_car_lng",
            value: pay.location.lng,
          },
        };
        break;
      // Special treatment for boolean field
      case "brake":
        obj = {
          ...acc,
          [pay.name]: {
            name: pay.name,
            value: pay.value ? 1 : 0,
          },
        };
        break;
      // Default treatment
      default:
        obj = {
          // the accumulator (acc) is used to keep the list already built and // introduce a new element (variable) at each pass
          ...acc,
          // The key is the variable alias
          [pay.name]: {
            // variable name name: pay.name, // variable value
            value: pay.value, // variable unit unit: pay.unit
          },
        };
    }
    return obj;
  }, {});
  // returns the new JSON object converted
  return newPayload;
}

// ***************************************************************************
// Main (basic default)
// ***************************************************************************

// converts the received payload into Kora standard
response = convertKora(payload);

Testing​

Now that our function is complete on the JS editor, we just need to also update the “Test Payload” editor with the aforementioned example so as to perform our first test. Do not forget to save it and then run it through the blue button on the top right-hand side of the screen:

Testing - Step 1

Once successfully ran, the following output on the right pattern is shown:

{
  "speed": {
    "name": "speed",
    "value": 35,
    "unit": "mhp"
  },
  "rpm": {
    "name": "rpm",
    "value": 2190
  },
  "fuel": {
    "name": "fuel",
    "value": 97,
    "unit": "%"
  },
  "location_car_lat": {
    "name": "location_car_lat",
    "value": -45.12356
  },
  "location_car_lngt": {
    "name": "location_car_lng",
    "value": -44.95116
  },
  "gear": {
    "name": "gear",
    "value": 2
  },
  "brake": {
    "name": "brake",
    "value": 0
  }
}

We can notice that each variable is separated by a different element. According to the Kora accepted standard, the element key (alias), as well as name and value are mandatory fields. Unit field is optional.

That is it! Our first function is up and running! Play around with results by changing input data, rerunning, and comparing changes on the output.

Editing functions​

In order to edit an existing function on your library, just click on the function’s name in the menu on the left-hand side of the screen.

This will open the “Function Editor” mode with the data of the requested function. After processing the necessary changes, make sure those have been saved.

Attention: whenever changing a function that is running, i.e., which is associated to a device, the result will be immediate. The best thing to do in that case is to create a temporary function with a different name, copy the original code of the function that is running, change it and test it. After everything is validated, copy the changed code back into the original function.

Deleting functions​

It is also possible to permanently delete functions from your library. To do that, click on the recycle bin button on the function card, which is on the table of contents screen of your library.

Attention: be careful as this command cannot be reversed.

Creating a more advanced function​

Now that we have created our first function, we can move to a bigger challenge.

Input​

Now we will process the data received from a LoRaWAN device but, for this time, with only one variable sent by transmission. The variable sent will follow the text pattern below:

“alias”:value (Ex: 'U12':220.0)

The alias is the summarized variable “nickname”. In that case, we would also need the variable description so that the system can be more complete. For that reason, we will create a complementary table within the function that can provide us, starting from the alias, with the description and unit of each variable received.

// Information table (unit and description) by variable alias
const units_descriptions = {
  u0: {
    unit: "V",
    description: "Three-phase voltage",
  },
  u12: {
    unit: "V",
    description: "Phase/Phase Voltage U12",
  },
  u23: {
    unit: "V",
    description: "Phase/Phase Voltage U23",
  },
  u31: {
    unit: "V",
    description: "Phase/Phase Voltage U31",
  },
};

As input JSON (Test Payload), we will use the following example, with the variable inside the payload field:

{
  "type": "uplink",
  "meta": {
    "network": "a653ee9da56a43b9ab308c3b2f06eb81",
    "packet_hash": "2d9d6158ddecaad4af2e23b86cf26386",
    "application": "0102030405060708",
    "device_addr": "167fd21a",
    "time": 1626108554.95,
    "device": "0012f80000001cc6",
    "packet_id": "96b8f16b3d723d3699e3b1140bf823aa",
    "gateway": "80029c09dca20000"
  },
  "params": {
    "rx_time": 1626108554.9121702,
    "port": 125,
    "duplicate": false,
    "radio": {
      "delay": 0.0039858818,
      "datarate": 4,
      "modulation": {
        "bandwidth": 125000,
        "type": "LORA",
        "spreading": 8,
        "coderate": "4/5"
      },
      "hardware": {
        "status": 1,
        "chain": 1,
        "tmst": 772967260,
        "snr": 10.5,
        "rssi": -85,
        "channel": 6
      },
      "time": 1626108554.9121702,
      "freq": 916.4,
      "size": 29
    },
    "counter_up": 8,
    "lora": {
      "header": {
        "class_b": false,
        "confirmed": false,
        "adr": true,
        "ack": false,
        "adr_ack_req": false,
        "version": 0,
        "type": 2
      },
      "mac_commands": []
    },
    "payload": "IlUxMiI6MjIwLjAw",
    "encrypted_payload": "H+q5NlkLN0z4z/hIxefLBw=="
  }
}

Conversion Function​

Our conversion function now is very different from the previous one. We only have one variable to handle, however, it is base64 encoded. This is a standard encoding used on the “LoRaWAN” protocol and for testing and knowledge purposes, you can decode the value received on the website: https://www.base64decode.org.

Look at the conversion of the data received in that example:

“data”: “IlUxMiI6MjIwLjAw” -> base64 decoder -> “U12”:220.00

This conversion is performed through the atob(payload.params.payload) function.

// srt -> base64 decoded payload (data)
const str = atob(payload.params.payload);

The lines that follow are responsible for separating the alias (grandezaEletrica) and the value (grandezaValor),besides an array with the complementary information of the variable (infos).

// grandezaEletrica -> varable alias
const grandezaEletrica = str.slice(1, str.indexOf(":") - 1);
// grandezaValor -> variable value
const grandezaValor = str.slice(str.indexOf(":") + 1, str.length);
// infos -> variable unit
const infos = units_descriptions[grandezaEletrica.toLowerCase()];

At the end, we set up the JSON object with the variable received. We also collected other two existing in the LoRaWAN protocol and converted them into variables as well.

// create JSON object for response
obj = {
  // variables decoder
  // varible alias
  [grandezaEletrica]: {
    name: infos.description, // variable name
    value: parseFloat(grandezaValor), // variable value
    unit: infos.unit, // variable unit
  },
  // LoRaWAN RSSI
  ["RSSI"]: {
    name: "RSSI",
    value: payload.params.radio.hardware.rssi,
  },
  // LoRaWAN SNR
  ["SNR"]: {
    name: "SNR",
    value: payload.params.radio.hardware.snr,
  },
};

Those two new variables bring interesting data (called metadata) about the network operation conditions:

  • RSSI : Received Signal Strength Indication -- SNR : Signal-to-Noise Ratio

Likewise, any other existing data in LoRaWAN protocol may be converted in exportation variable on the output JSON.

Full function (comment)​

/* *********************************************************************
// Every payload decoding routine has 2 parameters
// @param {String} payload - Payload sent by device
// @param {Object} response - Decoded payload for response
// ********************************************************************* */

// Information table (unit and description) by variable alias
const units_descriptions = {
  u0: {
    unit: "V",
    description: "Three-phase voltage",
  },
  u12: {
    unit: "V",
    description: "Phase/Phase Voltage U12",
  },
  u23: {
    unit: "V",
    description: "Phase/Phase Voltage U23",
  },
  u31: {
    unit: "V",
    description: "Phase/Phase Voltage U31",
  },
};

// ***************************************************************************
// Example function to transform original payload to Kora format
// Receives a JSON object with the full payload originally received
// In this case, a base64 data payload with 1 variable will be received
// in format 'alias':value (Ex: 'U12':220.0)
// The new converted JSON object will be returned by the function
// The response format must be::
// {
//   "u12": {
//      "name": "Tensão Fase/Fase U12",
//      "value": 220.0,
//      "unit": "V"
//   }
//}

// ***************************************************************************
function convertKora(payload) {
  // srt -> base64 decoded payload (data)
  const str = atob(payload.params.payload);
  // grandezaEletrica -> varable alias
  const grandezaEletrica = str.slice(1, str.indexOf(":") - 1);
  // grandezaValor -> variable value
  const grandezaValor = str.slice(str.indexOf(":") + 1, str.length);
  // infos -> variable unit
  const infos = units_descriptions[grandezaEletrica.toLowerCase()];
  var obj;
  // create JSON object for response
  obj = {
    // variables decoder
    // varible alias
    [grandezaEletrica]: {
      name: infos.description, // variable name
      value: parseFloat(grandezaValor), // variable value
      unit: infos.unit, // variable unit
    },
    // LoRaWAN RSSI
    ["RSSI"]: {
      name: "RSSI",
      value: payload.params.radio.hardware.rssi,
    },
    // LoRaWAN SNR
    ["SNR"]: {
      name: "SNR",
      value: payload.params.radio.hardware.snr,
    },
  };
  return obj;
}

// ***************************************************************************
// Main (basic default)
// ***************************************************************************

// converts the received payload into Kora standard
response = convertKora(payload);

Testing​

After succesfully run, we will have the following output on the correct pattern:

{
  "U12": {
    "name": "Phase/Phase Voltage U12",
    "value": 220,
    "unit": "V"
  },
  "RSSI": {
    "name": "RSSI",
    "value": -85
  },
  "SNR": {
    "name": "SNR",
    "value": 10.5
  }
}

Associating a function with a device​

Now that you already know how to build a customized function for data processing, you just need to associate it to one (or various) existing device on your account so that it can be automatically used on the pre-processing of each message received.

To do so, go to “Devices” menu, find the desired device and click on its name. Once inside the device, click on the “Decoder” tab.

Now you just need to choose the function (within the list of available functions on your library) that you want to associate with this device.

Function Device - Step 1

That is it! Your device will now have all received messages automatically processed by the new function.

Disassociating a function​

To delete a function associated with a device, you just need to go to the “Decoder” tab again, within the details of the device in question, and click on “X” on the right-hand side of the function name to which it is currently associated.