Decoding formatter in TTN

Hi,
I need some help with regards to javascript decoding please,
If my device is sending in the following format below , how would i declare those variables and bytes?
function Decoder(bytes, port) {
Possible data per measurement:

  • Message info (message type and description of the payload): Describes the configuration of the payload. Could be left blank (not recommended) when custom decryption-code is created. This example needs this value.
  • dB(A) fast: dB(A) measurement with fast response
  • dB(A) slow: dB(A) measurement with slow response
  • dB(C) fast: dB(C) measurement with fast response
  • dB(C) slow: dB(C) measurement with slow response
  • Leq(A): dB(A) average over set Leq time. Between the current measurement and current measurement time minus Leq time
  • Leq(C): dB(C) average over set Leq time. Between the current measurement and current measurement time minus Leq time
  • Positive Peak Hold A: The highest dB(A) value in the period between previous measurement and current measurement
  • Negative Peak Hold A: The lowest dB(A) value in the period between previous measurement and current measurement
  • Positive Peak Hold C: The highest dB(C) value in the period between previous measurement and current measurement
  • Negative Peak Hold C: The lowest dB(C) value in the period between previous measurement and current measurement
  • Battery voltage: The battery voltage
  • Location (latitude/longitude): The geolocation from onboard GPS
  • Timestamp of first sample: Only applies if multiple measurements per LoRa-message is configured
  • Timestamp of last sample: Only applies if multiple measurements per LoRa-message is configured

Note: The timestamp can be extracted from the uplink message from your provider (if they send it), but this is not so accurate. If you use the timestamp from the uplink message, you have to know the interval between the samples (for example hard-coded or from a database). Therefore we recommend to use the onboard GPS in combination with the 2 timestamp datatypes.

The payload of the example configuration is 320 bits (40 bytes) long (the payload description is also shown in the App):

| message info (16) | dB(A)f (10) | dB(A)s (10) | dB(C)f (10) | dB(C)s (10) | Leq(A) (10) | Leq(C) (10) | Pos.Peak(A) (10) | Neg.Peak(A) (10) | Pos.Peak(C) (10) | Neg.Peak(C) (10) | dB(A)f (10) | dB(A)s (10) | dB(C)f (10) | dB(C)s (10) | Leq(A) (10) | Leq(C) (10) | Pos.Peak(A) (10) | Neg.Peak(A) (10) | Pos.Peak(C) (10) | Neg.Peak(C) (10) | Bat (8) | Lat (32) | Lon (32) | first timestamp (16) | last timestamp (16) |

The ‘message info’ contains the type of the message and the configuration, what makes ‘autonomic’ parsing possible:

bit index description
0 Message type (2 bits)
1 Message type (2 bits)
2 dB(A) fast
3 dB(A) slow
4 dB(C) fast
5 dB(C) slow
6 Leq(A)
7 Leq(C)
8 Positive Peak Hold A
9 Positive Peak Hold C
10 Negative Peak Hold A
11 Negative Peak Hold C
12 Battery voltage
13 Location
14 Timestamp of first sample
15 Timestamp of last sample

Would it be fair to say this post is “I don’t know JavaScript, please help me write it” - the answer to this would inform the response - as in JavaScript 101 or just some direction on potential implementation.

The majority looks like a copy & paste from a device manual - what is this device, maybe there is a payload formatter for it already.

PS, formatted post for readability.

Hi, Yes you are 100% correct, any help would be awesome, i dont understand the bits and bytes and shifting.
The device is a Dutch Sensor system , its an audio monitoring unit with lora, so needing to get payload sorted before sending to ubidots. i am trying something like this,

function Decoder(bytes, port) {
var messagetype = (bytes[0] << 8);
var battery = (bytes[1] << 8);
var latitude = (bytes[2] << 32);
var longatude = (bytes[3] << 32);

return {
messagetype : messagetype,
battery: battery/ 10,
latitude: latitude,
longatude : longatude
}
}

function Decoder(bytes, port) {
  var messagetype = (bytes[0] << 8);
  var battery      = (bytes[1] << 8);
  var latitude     = (bytes[2] << 32);
  var longatude    = (bytes[3] << 32);
  
  return {
    messagetype : messagetype,
    battery: battery/ 10, 
    latitude: latitude,
    longatude : longatude
  }
}

Still not able to search for a Payload Formatter based on this level of detail.

If one is already written, you get it working this afternoon.

If you need it writing for you, that’s down to the availability of a volunteer to give it a go.

thank you for your feedback , happy to pay someone a few $ .

What make and model?

Without knowing what device, I suspect you’ll be looking at multiple $.

Or if you tell us which device, one of us may already have the payload formatter so no $.

device is IOt Sound meter as per website link
https://www.iotsoundsensor.com/wp-content/uploads/2021/11/310_payload_parser_manual.html
for someone who know bytes and bits and how to strip out to get in json it would be very easy.

Strangely, when I suggest that to a tradesman “oh, it should be a simple job for a good plumber/electrician/automechanic”, the bill increases.

So, having finally given us the link which in fact has the decoder built in and told us how easy it is, what party forfeit are you going to pay. In good faith, here’s the encoder from their website:


    function decodeUplink(e) {

        let payload = hexToBytes(input.value);

        var decoded_payload;
        switch (payload[0] >> 6) {
        case 0x00:
            // MEASUREMENT WITHOUT PAYLOAD DESCRIPTION
            break;
        case 0x01:
            // MEASUREMENT WITH PAYLOAD DESCRIPTION
            decoded_payload = parseWithDescription(payload);
            break;
        case 0x02:
            // RECEIVED SETTINGS (response of the settings request downlink)
            break;
        case 0x03:
            // RECEIVED ACK (CRC) (response of the settings update downlink)
            break;
        }

        result.innerHTML += JSON.stringify(decoded_payload) + "\n";
        console.log(decoded_payload)
    }

    function parseWithDescription(payload) {
        let payloadIndex = 2; // Skip the message type and info bytes

        let payloadPackage = {
            bat: 0,
            location: null,
            transmitInterval: null,
            measurements: []
        };

        const info = payload[0] << 8 | payload[1];
        const batSet = (info & 0x2000) !== 0;
        const locSet = (info & 0x1000) !== 0;
        const firstTsSet = (info & 0x0800) !== 0;
        const lastTsSet = (info & 0x0400) !== 0;

        //
        // Calculate sample count
        const skip = payloadIndex + (batSet ? 1 : 0) + (locSet ? 8 : 0) + (firstTsSet ? 2 : 0) + (lastTsSet ? 2 : 0);
        const totalBitsPerSample = numberOfBitsSet(info) * 10;
        const totalBitsInPayload = (payload.length - skip) * 8;
        const remainder = totalBitsInPayload % totalBitsPerSample;
        const sampleCount = (totalBitsInPayload - remainder) / totalBitsPerSample;

        if (sampleCount === 0) {
            return null;
        }

        //
        // Parse battery voltage
        payloadPackage.bat = batSet ? payload[payloadIndex++] / 10 : 0.0;

        //
        // Parse latitude and longitude
        if (locSet)
        {
            payloadPackage.location =
            {
                latitude: toNumber(payload.slice(payloadIndex, payloadIndex += 4)) / 1e7,
                longitude: toNumber(payload.slice(payloadIndex, payloadIndex += 4)) / 1e7
            };
        }

        //
        // Parse the timestamps
        let firstTimestamp = null;
        let lastTimestamp = null;
        let sampleInterval = 0;

        if (firstTsSet && lastTsSet) {
            const firstTS = payload[payloadIndex++] | payload[payloadIndex++] << 8;
            const lastTS = payload[payloadIndex++] | payload[payloadIndex++] << 8;

            if (firstTS != 0xFFFF) {
                // Timestamps available
                firstTimestamp = parseTimestamp(firstTS);
                lastTimestamp = parseTimestamp(lastTS);
                sampleInterval = (lastTimestamp - firstTimestamp) / (sampleCount > 2 ? sampleCount - 1 : 1);
                payloadPackage.transmitInterval = sampleInterval * sampleCount
            }
            else {
                // Parse the transmit interval to millis (HHHHMMMMMMSSSSSS)
                let hMillis = (lastTS >> 12) * 60 * 60 * 1000;
                let mMillis = ((lastTS >> 6) & 0x3F) * 60 * 1000;
                let sMillis = lastTS & 0x3F * 1000;

                payloadPackage.transmitInterval = hMillis + mMillis + sMillis;

                // Calculate the first timestamp
                sampleInterval = payloadPackage.transmitInterval / sampleCount;
                firstTimestamp = new Date() - payloadPackage.transmitInterval + sampleInterval;
            }
        } else if (firstTsSet && !lastTsSet) {
            // Not recommended
            const firstTS = payload[payloadIndex++] | payload[payloadIndex++] << 8;
            lastTimestamp = new Date();

            if (firstTS != 0xFFFF)// Timestamp present
            {
                firstTimestamp = ParseTimestamp(firstTS);
                sampleInterval = (lastTimestamp - firstTimestamp) / (sampleCount > 2 ? sampleCount - 1 : 1);
                payloadPackage.transmitInterval = sampleInterval * sampleCount
            }
            else {
                // first timestamp unknown!
                payloadPackage.transmitInterval = 60000; //WARNING! Unknown transmit interval!
                sampleInterval = payloadPackage.transmitInterval / sampleCount;
                firstTimestamp = lastTimestamp - payloadPackage.transmitInterval + sampleInterval;
            }
        }
        else if (!firstTsSet && lastTsSet) {
            // Not recommended
            const lastTS = payload[payloadIndex++] | payload[payloadIndex++] << 8;

            // Has the GPS found a fix?
            if (payloadPackage.location.latitude != 0 && payloadPackage.location.longitude != 0) {
                lastTimestamp = parseTimestamp(lastTS);
                payloadPackage.transmitInterval = 60000; //WARNING! Unknown transmit interval!

                // Calculate the first timestamp
                sampleInterval = payloadPackage.transmitInterval / sampleCount;
                firstTimestamp = lastTimestamp - payloadPackage.transmitInterval + sampleInterval;
            }
            else {
                // Parse the transmit interval to millis (HHHHMMMMMMSSSSSS)
                let hMillis = (lastTS >> 12) * 60 * 60 * 1000;
                let mMillis = ((lastTS >> 6) & 0x3F) * 60 * 1000;
                let sMillis = lastTS & 0x3F * 1000;

                payloadPackage.transmitInterval = hMillis + mMillis + sMillis;

                // Calculate the first timestamp
                sampleInterval = payloadPackage.transmitInterval / sampleCount;
                firstTimestamp = new Date() - payloadPackage.transmitInterval + sampleInterval;
            }
        }
        else {

        }

        //
        // Parse the samples
        let bitArray = toBitArrray(payload.slice(payloadIndex, payload.length));
        let index = 0;
        for (let i = 0; i < sampleCount; i++) {

            let measurement = {
                LAf: 0,
                LAs: 0,
                LCf: 0,
                LCs: 0,
                LAeq: 0,
                LCeq: 0,
                LAmax: 0,
                LAmin: 0,
                LCmax: 0,
                LCmin: 0,
                timestamp: ''
            };

            if (info & 0x0200)
            {
                measurement.LAf = parseMeasurement(bitArray.slice(index, index += 10));
            }
            if (info & 0x0100)
            {
                measurement.LAs = parseMeasurement(bitArray.slice(index, index += 10));
            }
            if (info & 0x0080)
            {
                measurement.LCf = parseMeasurement(bitArray.slice(index, index += 10));
            }
            if (info & 0x0040)
            {
                measurement.LCs = parseMeasurement(bitArray.slice(index, index += 10));
            }
            if (info & 0x0020)
            {
                measurement.LAeq = parseMeasurement(bitArray.slice(index, index += 10));
            }
            if (info & 0x0010)
            {
                measurement.LCeq = parseMeasurement(bitArray.slice(index, index += 10));
            }
            if (info & 0x0008)
            {
                measurement.LAmax = parseMeasurement(bitArray.slice(index, index += 10));
            }
            if (info & 0x0004)
            {
                measurement.LAmin = parseMeasurement(bitArray.slice(index, index += 10));
            }
            if (info & 0x0002)
            {
                measurement.LCmax = parseMeasurement(bitArray.slice(index, index += 10));
            }
            if (info & 0x0001)
            {
                measurement.LCmin = parseMeasurement(bitArray.slice(index, index += 10));
            }

            measurement.timestamp = new Date(firstTimestamp).toString();
            payloadPackage.measurements.push(measurement);

            firstTimestamp += sampleInterval; // Timestamp of the next sample.
        }

        return payloadPackage;
    }

    function numberOfBitsSet(info) {
        let units = info & 0x03FF;
        let count = 0;
        while (units !== 0) {
            count += units & 1;
            units >>= 1;
        }
        return count;
    }

    function toNumber(bytes) {
        let i = 0;
        for (let x = 0; x < bytes.length; x++) {
            i |= +(bytes[x] << (x * 8));
        }
        return i;
    }

    function toBitArrray(bytes) {
        let bitArray = [];
        for (let x = 0; x < bytes.length; x++) {
            for (var b = bytes[x], i = 0; i < 8; i++) {
                bitArray.push((b & 0x80) !== 0 ? 1 : 0);
                b *= 2;
            }
        }
        return bitArray;
    }

    function parseTimestamp(value) {
        // format HHHHHMMMMMMSSSSSS  12h notation
        var hours = value >> 12;
        var minutes = (value >> 6) & 0x3F;
        var seconds = value & 0x3F;
        var utc = new Date();

        if (utc.getHours() < hours) {
            hours = 23 - (11 - hours); // midnight (0 < 11)
            utc.setDate(utc.getDate() - 1);
        }
        else {
            hours = utc.getHours() > 12 && hours < 12 ? hours + 12 : hours; // Convert to 24h notation
        }
        return Date.parse(utc.getFullYear() + '-' + zeroPad(utc.getMonth() + 1) + '-' + zeroPad(utc.getDate()) + 'T' + zeroPad(hours) + ':' + zeroPad(minutes) + ':' + zeroPad(seconds));
    }

    function zeroPad(str) {
        str = str.toString();
        return str.length === 1 ? '0' + str : str;
    }

    function parseMeasurement(bits) {
        var cnt = 1;
        var value = 0;
        for (var i = 0; i < 10; i++) {
            if (bits[9 - i]) {
                value += cnt;
            }
            cnt += cnt;
        }
        return (value / 10 + 30).toFixed(1); // Don't forget the offset of 30dB!
    }

    function hexToBytes(hex) {
        for (var bytes = [], c = 0; c < hex.length; c += 2) {
            bytes.push(parseInt(hex.substr(c, 2), 16));
        }
        return bytes;
    }

Why is the supplier not able to do this for you for the world’s largest LoRaWAN network?

Hi, thank you for the reply, i guess you are right.
i agree the supplier should be doing it.
Thank you for extracting that code. very much appreciate it.