Sorry for the basic question but - how can i decrypt message with “Payload Functions” on console page? I mean if i send “12345” or “Hello” then paylod in Application data page shows “31 32 33 34 35” for 12345 and “48 65 6C 6C 6F” for Hello. If i enter any of those two payloads the results is in both cases empty. I’ve searched through forum but i did not find an answer. Can someone be so nice and explain me this results?
Tkanks,
Roberto
The data you’re seeing has already been decrypted by TTN and are hexadecimal representations of the decrypted “binary data”, each two digits being one “byte”, just like sent by your node. The bytes could be anything: like numbers, text, some special encoding of the states of some switches, and so on. And any combination. You’ll need to decode that, to reverse the encoding that your node used.
In your example, that binary data happens to be plain ASCII text (readable text; not recommended, as it uses a lot of bandwidth).
So, in your case each byte is one text character: hexadecimal 0x48 (which is the same as decimal 72, and is written with the 0x prefix to make clear it’s not the decimal number 48) is the character H. Likewise, 0x65 is e, 0x6c is l, and so on. For the number, 0x31 is the character 1, 0x32 is the character 2, and so.
The above needs 5 bytes to send the number 12345 as text. That’s a lot of waste, as by design numbers only have digits 0 to 9, and possibly one decimal point and one sign, while ASCII text also reserves space for upper case and lower case letters, punctuation characters, and so on.
When knowing you only need a number, then when sending in a dedicated binary format you would, for example, only need 16 bits (2 bytes) for a so-called “short signed integer”, which can hold all numbers between -32,767 and +32,767. And a 32 bits (4 bytes) integer (a.k.a. “long”) fits all numbers between -2,147,483,647 and +2,147,483,647, which when sent as characters (without the commas) would need as many as 11 bytes for one number.
If, for testing, you really want to send plain text and decode that into a readable format, then there are many online converters, or you’d need a payload function:
function Decoder(bytes, port) {
// Decode plain text; for testing only
return {
myTestValue: String.fromCharCode.apply(null, bytes)
};
}
After that, see https://github.com/TheThingsNetwork/workshops#send-sensor-data to learn how to send the data in better format.
Thank you a lot, you have really clarified some basic facts. For this purpose is LoraEncoder function - for example to change measured value to number representation of this value, isn’t it?
To be sure that this is the right way: way from measured Temp=23.45 to 0x09 0x29 on the TTN Application data page with as small payload as it possible is (from Arduino):
float Temp = 23.45;
LoraEncoder encoder(buffer);
encoder.writeTemperature(Temp);
LMIC_setTxData2(1, buffer, strlen((char *)buffer), 0);
It is not clear to me how Decoder in Payload Function shows the right value for the temperature but the field in Data page showing me {“celcius”:105.05} in the same time. Did i done something wrong? Your help is really appreciated.
Thank you,
Roberto
There are many ways to encode a value, especially when it comes to sending negative values and the order of the bits and bytes (Most or Least Significant Bit/Byte First; see MSB and endianness). For your test value, the order is the problem.
You’re seeing the result of the following calculation, with << being a left shift of 8 positions, which is basically the same as multiplying by 28, which is 256:
0x09 + (0x29 << 8) = 0x2909
In hexadecimal notation, 0x29 << 8 yields 0x2900, to which adding 0x09 results in 0x2909. Or, represented using decimal values: 9 + 41 × 256 = 10505 (and then divided by 100 to get your {"celcius": 105.05}).
However, you expected:
0x29 + (0x09 << 8) = 0x0929
…or: 41 + 9 × 256 = 2345.
So, somewhere along the line the 0x09 and 0x29 were swapped, either when encoding or when decoding.
If you’re using GitHub - thesolarnomad/lora-serialization: LoraWAN serialization/deserialization library for The Things Network to encode, then you should also use its decoder functions to reverse the operation in the payload functions. And you should also test with negative values.
Alternatively, similar to the TTN workshop example (which uses the Arduino highByte and lowByte), encode as:
float temp = 23.45; // Something like 0x41bb999a in IEEE 754 floating-point
int16_t celciusInt = temp * 100; // convert to signed 16 bits integer: 0x0929
uint8_t buffer[2]; // reserve 2 bytes in memory
// Handle high byte (MSB) first; 0x09 for 23.45
// 0x0929 >> 8 shifts the 0x29 out of memory, leaving 0x0009
// 0x0009 does not fit in a single byte, so only 0x09 is stored in buffer[0]:
buffer[0] = celciusInt >> 8;
// Handle low byte (LSB) next; 0x29 for 23.45
// 0x0929 does not fit in a single byte, so only 0x29 is stored in buffer[1]:
buffer[1] = celciusInt;
// Send on port 1, without asking for confirmation:
LMIC_setTxData2(1, buffer, sizeof(buffer), 0); // 0x0929 for 23.45
The workshop example (erroneously) decodes as:
var celciusInt = bytes[0] << 8 | bytes[1];
Here, | is the bitwise OR, which in this calculation (and with the proper grouping parentheses as, unlike the bitwise OR, addition has a higher precedence than the shift operator) is effectively the same as:
var celciusInt = (bytes[0] << 8) + bytes[1];
…or:
var celciusInt = (bytes[0] * 256) + bytes[1];
All are fine for positive numbers, but will fail for negative numbers as JavaScript’s bitwise operators always work with 32 bits and hence expect 4 bytes, where for negative numbers the missing bytes should become 0xFF (sign extension).
So, to also allow for negative numbers (such as 0xF6D7 for -2345, where 2345 is 0x0929), decode using:
// Test using 0x0929 for 2345, or 0xF6D7 for -2345
function Decoder(bytes, port) {
// Sign-extend 16 bits to 32 bits
// Fewer parentheses needed when using bitwise OR rather than addition
var celciusInt = (bytes[0] & 0x80 ? 0xFFFF<<16 : 0) | bytes[0]<<8 | bytes[1];
return {
celcius: celciusInt / 100
};
}
Above, the ternary operator in the code translates to: if the leftmost 8th bit of the first byte is set, then start the result with 0xFFFF, otherwise with 0x0000.
Alternatively, shift byte[0] 16 bits too far to the left, and then shift it back, which will do the sign extension on the fly, as the bitwise operator >> is the sign-propagating right shift:
var celciusInt = (bytes[0]<<24) >> 16 | bytes[1];
…which can even be written as:
var celciusInt = bytes[0]<<24>>16 | bytes[1];
(I know this might be a lot of information if this is new to you, I hope the links help… Not tested.)
@arjanvanb Thank you for your clear explanation. In meantime i have fried my LoraBee so i had to wait for new one 
I understand coding process but decoding with decoder on TTN page is not yet clear to me.
How can i use methods from https://github.com/thesolarnomad/lora-serialization ( for example uint16(bytes.slice(x, x+2)) in Payload functions?
Thank you,
Roberto
I’ve not used the library, but its documentation states:
Example
Paste everything from src/decoder.js into the decoder method and use like this:
function (bytes) { // code from src/decoder.js here return decode(bytes, [latLng, unixtime], ['coords', 'time']); }
That uses the old syntax of v1 Staging. So I’d say: for v2 Production copy all of its decoder JavaScript into the function Decoder(bytes, port) function:
function Decoder(bytes, port) {
// First, copy ALL CODE from
// https://github.com/thesolarnomad/lora-serialization/blob/master/src/decoder.js
var bytesToInt = function(bytes) {
var i = 0;
...
humidity: humidity,
latLng: latLng,
decode: decode
};
}
// Next, use it; test with 9d5b 64a6fafd6a240409
// for 23453, -33.905052, 151.26641
return decode(bytes, [uint16, latLng], ['val', 'coords']);
}
In the result, this will combine the latitude and longitude into a single array, named coords in the code above:
{
"coords": [-33.905052, 151.26641],
"val": 23453
}
You could also format the results:
var decoded = decode(bytes, [uint16, latLng], ['val', 'coords']);
return {
val: decoded.val,
lat: decoded.coords[0],
lng: decoded.coords[1]
};
…to get the JSON result:
{
"lat": -33.905052,
"lng": 151.26641,
"val": 23453
}
Alternatively, use the helper methods manually:
var i = uint16(bytes.slice(0, 2)); // see next posts for improved version
var c = latLng(bytes.slice(2, 10));
return {
val: i,
lat: c[0],
lng: c[1]
};
Or don’t use it at all (neither on the node nor in the payload functions), and see the example and links in my earlier answer.
@arjanvanb thank you - things become more clear. I have two more questions. The first one is regarding to size of declared array on Arduino. If i declare byte mydata[3]; than TTN Data shows me 2 received bytes. If i declare mydata[2]; than TTN Data shows me only one byte. Why?
The other question is regarding use of uint16(bytes.slice(x, x+2)); What is meant with argument x? If i use the method i receive Error(“int must have exactly 2 bytes”)
Thank you,
Roberto
Without seeing your code, my guess would be that you’re still sending text, as a null-terminated string. The maximum length of such string is always 1 less than its total length, as it always needs a place to store the 0x00 character.
If you look at the manual for slice, you’ll see it needs a start index and an end index. In the examples x is the start index. In my example:
var i = uint16(bytes.slice(0, 2)); // use bytes 0 and 1
var c = latLng(bytes.slice(2, 10)); // use bytes 2 thru 9
You could rewrite that to:
var x = 0;
var i = uint16(bytes.slice(x, x + 2));
x = x + 2;
var c = latLng(bytes.slice(x, x + 8));
x = x + 8;
...
And as x = x + 2 can be written as x += 2, you can combine this to:
var x = 0;
var i = uint16(bytes.slice(x, x += 2));
var c = latLng(bytes.slice(x, x += 8));
Here is the Arduino code:
byte mydata[3];
uint16_t k;
// Prepare upstream data transmission at the next possible time.
Serial.print("\nReady to send string mydata “);
Serial.print(k);
Serial.println(” in DEC is: ");
LoraEncoder encoder(mydata);
encoder.writeUint16(k); //
LMIC_setTxData2(1, mydata, sizeof(mydata) - 1, 0);
Serial.println(F(“Packet queued”));
Is it correct?
There’s your surprise…
This will fill 2 bytes:
encoder.writeUint16(k);
So, declaring memory space for just 2 bytes is good enough (you’re not sending a null-terminated string here, but just plain bytes that you encode yourself):
byte mydata[2]; // size is 2
...
LMIC_setTxData2(1, mydata, sizeof(mydata), 0);
Holy shit! - true! But for what purpose is -1 in this sketch https://github.com/matthijskooijman/arduino-lmic/blob/master/examples/ttn-abp/ttn-abp.ino (line 142)?
Well, that example does send a null-terminated string:
static uint8_t mydata[] = "Hello, world!";
Here, the text is 13 characters long, but the size of mydata will be 14, to add an 0x00 at the end. So, to only send the text without the trailing 0x00 character, sizeof(mydata) - 1 is needed.
The example would better use strlen((char *)mydata) to get the size up to the first 0x00 character. Or better yet: not send strings at all 
At the end of this topics i would like to thank you to @arjanvanb for the useful advices and i would like to make short digest to be useful for the someone else also.
Here is the highlights:
I hope this will make a day to someone else
When everything work fine comes another question: how can i decrypt messages from two different nodes where one sending for example temperature and the other sending temperature and voltage?
You might be able to determine the difference by the size of what is received.
Alternatively what we did once when we had all kinds of different nodes, we added a leading byte describing what data was following. I.e. 0x00 for temp 2 bytes (total of 3 bytes), 0x01 for temp 2 bytes plus voltage 1 byte. Etc.
You can use a different port for each message type.
Would you be so kind and give me an example how to do this?
For an example using the port number, see step 3 and 5 of “Getting temperature, humidity and battery level” in Getting Badgerboard to work with TTN .
For LMiC, the port number is the first parameter in:
LMIC_setTxData2(1, buffer, sizeof(buffer), 0);
In the TTN Arduino library, it’s the last parameter in:
ttn.sendBytes(buffer, sizeof(buffer), 1);
A bit more details in Is there any documentation on payload functions?
And, of course, one can also define multiple applications for different payloads.
Please help me with decoding data. I have this payload:
05 0A EF 22 28
where 050A is 2565 (this is temperature in celsius / 100)
where EF22 is 8944 (this is pressure in hPa, subtracted 900 and multiplied with 100)
where 28 is 40 (humidity in %)
This is decoder script but beggining is wrong.
function Decoder(bytes) {
var temperature = bytes[0];
var pressure = bytes[2];
var humidity = bytes[4];
dekodovana_teplota = temperature / 100;
dekodovany_tlak = (pressure / 100) + 900;
return {
teplota: dekodovana_teplota,
tlak: dekodovany_tlak,
vlhkost: humidity
}
}
Im getting this but first two values are bad.
{
"teplota": 0.34,
"tlak": 900.34,
"vlhkost": 40
}
Good values are:
{
"teplota": 25.65,
"tlak": 989.44,
"vlhkost": 40
}
The first two values use two bytes each. The decoder won’t automatically add the other bytes for you. So, you’d need:
// LSB (least significant byte first):
var temperature = bytes[1]<<8 | bytes[0];
var pressure = bytes[3]<<8 | bytes[2];
Also, to support negative temperatures, you’ll want to read my notes about sign-extension above, and use:
// LSB (least significant byte first) and sign-extension to get 4 bytes
// for proper decoding of negative values:
var temperature = bytes[1]<<24>>16 | bytes[0];