How to do proper number calculations in a Payload Format decoder?

Application Development
3

There are quite a few things going on here:

  • Like @cslorabox already noted: JavaScript is not the same as Java. The payload formats in TTN Console use JavaScript (ECMAScript 5 to be precise). See also MDN Web Docs to learn that.

  • JavaScript’s decimal numbers are not exact.

    For example, 0.1 + 0.2 yields 0.30000000000000004. See Here is what you need to know about JavaScript’s Number type. After reading that, you can avoid the unexpected extra precision by changing the order of your calculations, to force JavaScript to use exact integer numbers for the intermediate results:

    • 0x54C9D04F / 10000000 - 90 = 52.251220700000005

    • (0x54C9D04F - 900000000) / 10000000 = 52.2512207

  • Apparently your device’s code is converting the decimal coordinates into positive integer numbers. However, when using bitwise operators in your decoder, then JavaScript expects signed 32 bits numbers using two’s complement encoding, hence allowing for numbers from -2,147,483,648 through 2,147,483,647.

    The longitude ranges between -180 and 180 degrees. Now, when adding 180 to the longitude in the device’s code, and multiplying that by 10,000,000 to keep 7 decimals (which is unrealistic anyway), the expected values that the device sends for the longitude range from 0 through 3,600,000,000. That’s too large for a signed 32 bits integer:

    • In your example, the hexadecimal bytes 68C418C0 are converted into decimal 1,757,681,856, no problem there.

    • When using the maximum values for your use case, 1,800,000,000, 3,600,000,000 and 255, or the hexadecimal bytes 6B49D200 D693A400 FF, the longitude D693A400 will decode to the negative number -694,967,296, so the end result will show -249.4967296 instead of 180.0.

    • Likewise, any longitude larger than 34.7483647 will fail in your decoder!

    To mitigate that:

    • Tell JavaScript the intermediate result is unsigned, using the zero-fill right shift >>> along with actually shifting zero bits:

      // Force an unsigned 32 bits integer using zero-fill right shift:
decoded.latitude = (
  (bytes[0]<<24 | bytes[1]<<16 | bytes[2]<<8 | bytes[3])>>>0
    - 900000000) / 10000000;

decoded.longitude = (
  (bytes[4]<<24 | bytes[5]<<16 | bytes[6]<<8 | bytes[7])>>>0
    - 1800000000) / 10000000;

    • Or, don’t use bitwise operators, but use regular math:

      // Transmitted as an unsigned 32 bits integer; don't use bitwise operators
decoded.latitude = (
  (bytes[0] * 0x1000000 + bytes[1] * 0x10000 + bytes[2] * 0x100 + bytes[3])
    - 900000000) / 10000000;

decoded.longitude = (
  (bytes[4] * 0x1000000 + bytes[5] * 0x10000 + bytes[6] * 0x100 + bytes[7])
    - 1800000000) / 10000000;

    • Or, scale by a smaller value in the device’s code to always stay below the maximum of a signed integer (multiply by, say, 1,000,000 to keep 6 decimals, which is still unrealistically precise).

    • Or, simply send signed numbers (don’t add the 90 and 180 in the device’s code, and don’t subtract it in the decoder).

Finally, next time please don’t post screenshots for text: even after I cropped the image it’s hard to read on small devices, and it’s impossible to copy or quote. See How do I format my forum post? [HowTo] Also, I doubt the output shown for longitude goes with the code that is shown…?

2 Likes