Decoding a float value in a payload function

Application Development
1

I recently made my first node for the TTN network using this guide: How to build your first TTN node: Arduino + RN2483 and the sample code from https://github.com/jpmeijers/RN2483-Arduino-Library.

After sending some “Hello world” messages around, and seeing them come in in the dashboard, I connected a temperature sensor (SI7021) to it and started sending them out. I was first sending the values out as text (in the format xx.xx), but changed it to float, so that I could send it out as 4 bytes.

The one thing I can’t seen to figure out is how I should decode these 4 bytes in the payload function. I see that 78CE8D41 is coming in as payload, but I have no clue as to how I should convert that back into a float number.

I’ve seen suggestions like : var value = (bytes[0] + (bytes[1] << 8)); , but that is just giving me very large numbers. It should decode into something like 17.75

Can anyone point me in the right direction?

2

17.725815 :slight_smile:

3

https://www.h-schmidt.net/FloatConverter/IEEE754.html

You use little endian format, so don’t forget to reverse byte order (for instance, to decode 78CE8D41 you should enter 0x418dce78 to the form).

4

Just a thought looking at your use case:
Do you really need the second decimal in your temperature? The sensor only has an accuracy of +/-0.4 C.
Furthermore, for most applications one (or even none) decimal digits will suffice.

So you can save bytes/airtime :wink:

At least according to this datasheet I googled.

3 Likes
5

It seems that just one byte is enough. Measurement range 95 C ( -10 to +85), 0.4 C step… 95/0.4 = 237.5 < 256

1 Like
6

Thanks for all your replies and your patience with a newbie like me :slight_smile:

Good point about the decimals, a second decimal should not be needed, I would like to have 1 decimal precision however.
What https://www.h-schmidt.net/FloatConverter/IEEE754.html is doing is exactly what I would like to do in my payload function. I still can’t figure out how my payload function should look to do the same thing.

This is by the way my Arduino code, which largely is the same as the sample on https://github.com/jpmeijers/RN2483-Arduino-Library

In this code I’m still using 4 bytes.

#include <rn2xx3.h>
#include <SoftwareSerial.h>
#include <Wire.h>
#include "i2c.h"

#include "i2c_SI7021.h"
SI7021 si7021;

SoftwareSerial mySerial(10, 11); // RX, TX

//create an instance of the rn2xx3 library, 
//giving the software serial as port to use
rn2xx3 myLora(mySerial);

// the setup routine runs once when you press reset:
void setup() 
{
  Serial.begin(57600); //serial port to computer
  si7021.initialize();
  
  //output LED pin
  pinMode(13, OUTPUT);
  led_on();
  
  // Open serial communications and wait for port to open:
  mySerial.begin(9600); //serial port to radio
  Serial.println("Startup");

  initialize_radio();

  led_off();
  delay(2000);
}

void initialize_radio()
{
  //reset rn2483
  pinMode(12, OUTPUT);
  digitalWrite(12, LOW);
  delay(500);
  digitalWrite(12, HIGH);

  delay(100); //wait for the RN2xx3's startup message
  mySerial.flush();

  //Autobaud the rn2483 module to 9600. The default would otherwise be 57600.
  myLora.autobaud();

  //check communication with radio
  String hweui = myLora.hweui();
  while(hweui.length() != 16)
  {
    Serial.println("Communication with RN2xx3 unsuccesful. Power cycle the board.");
    Serial.println(hweui);
    delay(10000);
    hweui = myLora.hweui();
  }

  //print out the HWEUI so that we can register it via ttnctl
  Serial.println("When using OTAA, register this DevEUI: ");
  Serial.println(myLora.hweui());
  Serial.println("RN2xx3 firmware version:");
  Serial.println(myLora.sysver());

  //configure your keys and join the network
  Serial.println("Trying to join TTN");
  bool join_result = false;
  
  //ABP: initABP(String addr, String AppSKey, String NwkSKey);
  join_result = myLora.initABP("02017201", "8D7FFEF938589D95AAD928C2E2E7E48F", "AE17E567AECC8787F749A62F5541D522");
  
   while(!join_result)
  {
    Serial.println("Unable to join. Are your keys correct, and do you have TTN coverage?");
    delay(60000); //delay a minute before retry
    join_result = myLora.init();
  }
  Serial.println("Successfully joined TTN");
  
}

// the loop routine runs over and over again forever:
void loop() 
{
    led_on();

    static float  temp;
    si7021.getTemperature(temp);
    si7021.triggerMeasurement();

    Serial.print("TXing: ");
    Serial.println(String(temp));

    
    typedef union
    {
     float number;
     uint8_t bytes[4];
    } FLOATUNION_t;

    FLOATUNION_t myFloat;
    myFloat.number = temp;

    myLora.txBytes(myFloat.bytes, sizeof(myFloat.bytes));
    
    led_off();
    //wait 5 minutes
    delay(300000);
    
}

void led_on()
{
  digitalWrite(13, 1);
}

void led_off()
{
  digitalWrite(13, 0);
}
7

Depends on the language/platform that you are going to use for payload decoding :slight_smile:

8

Precision is not expressed in number of decimals!
See for example Precision, Accuracy, and Resolution

9

ah, I didn’t know I had a choice :slight_smile:
The only payload functions I saw where those in the dashboard (where you can create them for decoder/converter/validator)

I also ran the sample code available there to get the values back locally in nodeJs, but at that point, the data had already gone through the decoder function, so it seems I need to do the decoding there in what looks like node.js (javascript) code.

10

You are not required to do any decoding in the backend. You can get the ‘original’ data and decode it in your own code written in a programming language of your own choice. Decoding in the back end is just one of the options available to you.

11

Classes.Binary Parser - JSFromHell.com: JavaScript Repository may help.

1 Like
12

Took me some fiddling, but it seems like I figured it out. I converted my float value (that had 2 digits before the decimal, and 2 after it) into a 16bit integer value, by multiplying it by 100.

After that, I converted that value into 2 bytes.

The part of my arduino sketch to do that looks like this:

static float  temp;
si7021.getTemperature(temp);
si7021.triggerMeasurement();

int16_t int_temp = (int16_t)(temp * 100);
byte data[2];
data[0] = int_temp >> 8;
data[1] = int_temp & 0xFF;
    
myLora.txBytes(data, sizeof(data));

In my payload function, I than had to the the opposite:

function (bytes) {

var temperature = (bytes[0] << 8 | bytes[1]);

  return {
    temp: temperature/100.0
  };
}

I now get the same values back that I’m sending :slight_smile:
A good help to figure this out was this youtube video for me: https://www.youtube.com/watch?v=MBcKblLODJQ&t=4s

@kersing you mention that I don’t have to use the decoding function in the ttn backend, can you explain how that would work? When I remove the payload function, the default payload function seems to be put back in.

1 Like
13

@erikHouten
The default function does not decode data, it simply passes the data on to the client which means you can do your own decoding in the client.

14

a 16bit integer is 2 bytes and a 2 byte array is 16bits. Ie you don’t need to create the 2 byte array. Just send the 16bit int.

15

You might also want to have a look at https://github.com/thesolarnomad/lora-serialization (as mentioned before in Approaches for efficient payload encoding). It should take care of any endiannes issues.

16

var msg2 = { payload: msg.payload.length };
msg2.payload = JSON.parse(msg.payload);
msg2.payload = new Buffer(msg2.payload.payload, ‘base64’).toString(‘hex’);
var lat = Buffer(msg2.payload, ‘hex’).readFloatLE(0);
var lon = Buffer(msg2.payload, ‘hex’).readFloatLE(4);
msg2.payload= “[{“lat”:” + lat + “,“lng”:” + lon + “}]”;//"{“lat”:lat,“lng”:lon};
return msg2;

This is how I am decoding the ttn payload via node-red. This assumes that the payload I am getting from ttn via mqtt is an 8 byte payload, coming out of the node-red mqtt node in base64, with the first four bytes being one float, and the second four bytes being the second float.

If you are trying to decode a raw 8 byte value, it’s even easier:

var lat = new Buffer(msg.payload).readFloatLE(0);
var lon = new Buffer(msg.payload).readFloatLE(4);
msg.payload= “[{“lat”:” + lat + “,“lng”:” + lon + “}]”;//"{“lat”:lat,“lng”:lon};
return msg;

I hate javascript and am terrible at doing anything with it, so there is probably a more efficient way to do this.

1 Like
17

@erikHouten
On staging.thethingsnetwork.org/applications I use this function:

function(bytes) {

   var Bytes2Float32 = {
       decodeFloat: function(bytes, signBits, exponentBits, fractionBits, eMin, eMax, littleEndian) {
           var totalBits = (signBits + exponentBits + fractionBits);
           var binary = "";
           for (var i = 0, l = bytes.length; i < l; i++) {
               var bits = bytes[i].toString(2);
               while (bits.length < 8)
                   bits = "0" + bits;
               if (littleEndian)
                   binary = bits + binary;
               else
                   binary += bits;
           }
           var sign = (binary.charAt(0) == '1') ? -1 : 1;
           var exponent = parseInt(binary.substr(signBits, exponentBits), 2) - eMax;
           var significandBase = binary.substr(signBits + exponentBits, fractionBits);
           var significandBin = '1' + significandBase;
           var i = 0;
           var val = 1;
           var significand = 0;
           if (exponent == -eMax) {
               if (significandBase.indexOf('1') == -1)
                   return 0;
               else {
                   exponent = eMin;
                   significandBin = '0' + significandBase;
               }
           }
           while (i < significandBin.length) {
               significand += val * parseInt(significandBin.charAt(i));
               val = val / 2;
               i++;
           }
           return sign * significand * Math.pow(2, exponent);
       }
   }


   var value1 = [bytes[4], bytes[5], bytes[6], bytes[7]];
   var value2 = [bytes[10], bytes[11], bytes[12], bytes[13]];
   var value3 = [bytes[14], bytes[15], bytes[16], bytes[17]];
   var value1float = Bytes2Float32.decodeFloat(value1, 1, 8, 23, -126, 127, false);
   var value2float = Bytes2Float32.decodeFloat(value2, 1, 8, 23, -126, 127, false);
   var value3float = Bytes2Float32.decodeFloat(value3, 1, 8, 23, -126, 127, false);

   return {
       payload : bytes,
       batteryVoltage: value1float,
       pressure: value2float,
       temperature: value3float
   };
}

It works for me. The decoded json is sent to the Azure IoTHub (with webjobs), there it goes to a SQL-server and a demo-web app shows the data (http://cloud.keller-druck.com).

Be aware that a Test doesn’t work (

TypeError(“Cannot access member ‘toString’ of undefined”) )

On console.thethingsnetwork.org/applications on the other hand, you must have a successful Test before saving the payload function. Therefore, I couldn’t get it working there.

1 Like
18

Hi,

Can someone send me a code for HX711 loadcell amplifier and DHT22 library?

I have tried this:

uint16_t weight = scale.get_units() * 100; 

// Read sensor values and multiply by 100 to effictively have 2 decimals
uint16_t humidity = dht.readHumidity(false) * 100;

// false: Celsius (default)
// true: Farenheit
uint16_t temperature = dht.readTemperature(false) * 100;


// Split both words (16 bits) into 6 bytes of 8
byte message[6];

message[0] = highByte(temperature);
message[1] = lowByte(temperature);
message[2] = highByte(humidity);
message[3] = lowByte(humidity);
message[4] = highByte(weight);
message[5] = lowByte(weight);

LMIC_setTxData2(1, message, sizeof(message), 0);
}

On TTN side this is the payload function:

function (bytes) {

var decodedValue1 = (bytes[4] << 8) | bytes[5];
var decodedValue2 = (bytes[2] << 8) | bytes[3];
var decodedValue3 = (bytes[0] << 8) | bytes[1];

return { Weight: decodedValue1/100, Temp: decodedValue2/100, Hum: decodedValue3/100};
}

The result is this. It should be 4 decimal each:

Hum 20.3
Temp 1
Weight 654.36

19

You’re multiplying by 100, and storing that in memory as an “unsigned 16 bits integer” (uint16), hence losing all remaining decimals. For example: 12.34567 is stored and sent as 1234, which is probably just fine as most sensors really don’t have an accuracy of 4 decimals (or even 2 decimals). It even says this in your code:

// Read sensor values and multiply by 100 to effictively have 2 decimals
uint16_t humidity = dht.readHumidity(false) * 100;

As 2 bytes unsigned integers can only hold values 0 to 65535, you don’t really have room to multiply by, say, 1000 to get 3 decimals as then any value larger than 65.5 would cause an overflow with unexpected results.

Things are even more complicated if the values might be negative, in which case the JavaScript in the payload functions expects 4 bytes to do proper calculations for negative values:

// Signed 16 bits integer, -32767 up to +32767
int16_t temperature = dht.readTemperature(false) * 100;

…with in the payload function:

decodedValue3 = (bytes[0] & 0x80 ? 0xFFFF<<16 : 0) | bytes[0]<<8 | bytes[1];

More details on the above “sign-extension” in Decrypting messages for dummies (and more links to examples in Is there any documentation on payload functions?).

You’re also mixing the values. You’re sending 2 bytes for temperature, humidity and weight. But you’re are showing that as humidity, temperate and weight. So, 20.3 is your temperature. I’d clean up and use the same order in your code:

// For staging-v1
function(bytes) {
    // temperature might be negative; sign-extend to get 4 bytes:
    var t = (bytes[0] & 0x80 ? 0xFFFF<<16 : 0) | bytes[0]<<8 | bytes[1];
    var h = bytes[2]<<8 | bytes[3];
    var w = bytes[4]<<8 | bytes[5];

    return {
      temperature: t / 100,
      humidity: h / 100,
      weight: w / 100
    }
}

I don’t understand why you’re getting a humidity of exactly 1. What does the false in the formula do? I guess the result of dht.readHumidity(false) is 1 or 1.00, which multiplied by 100 is sent as 100 and then decoded back to 1.

By the way, on the production environment the first line of a Decoder payload function needs to read:

function Decoder(bytes, port) {

And as Staging will be taken offline at the end of March, I strongly advise to use production.

2 Likes
20

2 posts were split to a new topic: How to use the HX711 24 bit ADC for weight scales?