How to decode Elvaco CMi4110 standard M-Bus payload?

End Devices (Nodes)
1

Hello, i’ve installed a sensor and connected it with TTN. I do get my payload in this format:
000C06575800000C14223902000B2D5701000B3B2008000A5A06060A5E41040C789938187002FD170000

How can i decode it? I can read the payload for e.x.
U can “cut” the payload in small packages with informations:

0C06 57 58 00 00 ==> 00 00 5,8 57

0C14 22390200 ==> 000239,22

0B2D 570100 ==> 00015,7

0B3B 200800 ==> 000,820

0A5A 0606 ==> 060,6

0A5E 4104 = as reading example: the first 4 numbers do name the value and how many decimals the number has (in every package). This information is from the sensor manual. the last numbers is the value. In this case we do get a Temperature with 1 decimals (e.g. 15,1°C). To get the value of the temperature u have to read the payload backwards with two numbers each: 41 04 => 04 41 => 044,1°C (hopefully u do understand what i mean). Now i want to decode the payload in way the i get back: temperature: 44,1 etc. can u help me?

0C78 99381870 ==> serialnumber

02FD17 0000 ==> errorcode

1 Like
2

Hi log_win,
It helps to do some research yourself before asking others, like searching for ‘payload functions’.

Have a look at:

2 Likes
3

Thank you very much. I’ve aready saw it via google but i can not connect it to my payload. I am very new.

4

You need to read up on this, the physical payload carries lots of encoded data plus your own data - check out the applications and payloads

5

Writing a payload decoder can be a bit tricky, specially when you are not used to writing JS code. Some companies publish payload decoders for their products, but not all do. You can find a good example at the Elsys website. Some companies have decoders, but you need to register to download them. Others do not care for TTN.

We have been missing some function in JS (maybe this can be done better, please let me know), hopefully they can be a help for people, that have to start writing their own decoders. You can add the code above the decoder.

// read number from buffer
function readInt16(buffer, p) {
	return buffer[p+1] << 8 | buffer[p];
}

function readInt32(buffer, p) {
	return buffer[p+3] << 24 | buffer[p+2] << 16 | buffer[p+1] << 8 | buffer[p];
}

function readInt24(buffer, p) {
	return  buffer[p+2] << 16 | buffer[p+1] << 8 | buffer[p];
}

// Based on https://stackoverflow.com/a/37471538 by Ilya Bursov
// read float value from buffer 
  function readFloat32(bytes, p) {
    // JavaScript bitwise operators yield a 32 bits integer, not a float.
    // Assume LSB (least significant byte first).
    var bits = bytes[p+3]<<24 | bytes[p+2]<<16 | bytes[p+1]<<8 | bytes[p];
    var sign = (bits>>>31 === 0) ? 1.0 : -1.0;
    var e = bits>>>23 & 0xff;
    var m = (e === 0) ? (bits & 0x7fffff)<<1 : (bits & 0x7fffff) | 0x800000;
    var f = sign * m * Math.pow(2, e - 150);
    return Math.round(f*1000.0)/1000.0;
  }  

// test Bit, Pos = 0..7
function bit(byte, pos) {
	return (byte & (1<<pos))>0;
}
3 Likes
6

Hey @EFthings

thank you very much for your answer! For better understanding. Do you mean then code from the Elsys website to convert to my paylod? For What is the code you were posting in your comment?
Edit: when i use my payload on the elsys site i do get the output Temperature = 0 (which can not be)

7

Is it possible to slice the payload string? That would help me!

8

Please read the tutorials. If you set up a plain application, the payload ist just a byte buffer. For cenvenience you can install a payload decoder that converts your bytes into a JSON object. Most lora platforms provide some way to decode a payload, but there is no standard. So you need a uniqe converter for every platform.

For more information on converters, please refer to the links given above.

Maybe you set up your first devices and go through the installation process, that makes things clear.

9

Thanks for your information. My device was/is working and i do get a payload which need to be encrypted. I can read the payload by hand (see above). But how can i put it in a decoder? I was looking at the tutorials the whole weekend but i do not understand. Cryptic world for me and i am looking for some help and explenation. As a problem i do see the length of my payload?!
Edit: its a normal mbus systematic as i see

10

I remember that I got stuck at the same point when I was starting with loraWan. If you know how, it is simple…

So, If you get data, you have created an application and added some devices. You will need one application for every device type.

In the application panel you see a button “Payload formats”. You will need a “decoder” to “decrypt” your payload. If you do not have a decoder, your payload is sent as a byte buffer.

So, place some js-code here, like:

function Decoder(bytes, port) {
  // Decode an uplink message from a buffer
  // (array) of bytes to an object of fields.
  var decoded = {};

  if (port === 100) {
    decoded.open = bytes[0]&1;
    decoded.batt = (25+(bytes[1]&15))/10;
    decoded.temp = (bytes[2]&127)-32
    decoded.time = (bytes[4]<<8)|bytes[3];
    decoded.count = (bytes[7]<<16)|(bytes[6]<<8)|bytes[5];
    return decoded;
  }

}

Now you get your payload as a JSON-Object.

You should care for the port number, as most devices send different payloads for different ports. A port is just a number send by the device to quaify the kind of message.

To get your data out of TTN there are different ways. I have used node-red, which can use MQTT or a special TTN-Node.

11

@EFthings01
Hey thanks. Well i have my device installed up to the point “Payload Formats”. I do not have a written decoder to get the payload in JS (that is my problem!). Thats the point where i need some help. I can tell which part of payload is giving which information and how it should look but i can not translate it in a decoder to generate a JSON output.
Afterwards i am able to writer the JSON output via node red in an influxDB and make visuization with graphana :slight_smile: that is working fine.

Your code kinda works…
I guess the payload is a hex string which will be converted?! The payload did not need to be converted. I need to take a number from a payload.

E.G. 0A5E 4104 = you can Skip 0A5E, now you have to take it backwards 04+41=0441=44,1°C

Edit: and the port is knowing (port===2)

12

I think, the code above is easy to adapt and you can use the decoder functions given above.

Anyway, you can also leave the decoder empty and write a decoder in node-red. As both are JS, the code will be pretty similar.

Please be careful where you put your “return decoded”. If you place it inside the if-statement, messages will only get forwarded if they are translated. If you place it outside, you will get a JSON-Payload for port 100 and a binary buffer for all other messages. This may give some errors in node-red .

13

*0A5E 4104
04 = will be bytes[4]? If i do use it like that i dont get 04 as result

I am so sorry but for me everything is spanish… the tutorials are not very usefull if u have 0 knowledges about programming :o at least it feels like

14

A few observations:

  • You’re not telling us which device you’re using? Please tell us, so future users can find this post. Also please add a link to the documentation so people don’t need to guess to help you.

  • As for “I guess the payload is a hex string”: no. LoRaWAN devices basically transmit bits, a.k.a. binary data. In your payload 4104 is a hexadecimal representation of two bytes, each having 8 bits: 01000001 and 00000100. (Often such presentation is prefixed with 0x to indicate it’s shown as hexadecimal, like 0x4104.) But that is just a way to display the bytes. Most often, computers decode the above 2 bytes to the decimal number 16644. Please see How to send payload in Hex-format? and its links, like to Working with bytes, to make sure you understand.

  • It’s really weird to decode those hexadecimal numbers 0x04 and 0x41 to the decimal 44.1 °C. But we’ve seen that before in Decoding Smart Waste Bin Detector DF702 payload:

    (This encoding, when indeed true, makes handling unnecessary complex and makes the payload unnecessary long. The payload you’re showing is 42 bytes. The maximum at SF12 is 51 bytes; sending 42 will take a whopping 2.5 seconds.)

  • To use something like the above, you’d need to change the order, but above all you still need to find the specific bytes you need in the above calculation. You could try to find the hexadecimal pattern 0x0A5E, but the user manual should explain how to truly know how many bytes to skip.

  • As for:

    In programming, the “index” for arrays (lists) is often zero-based, so for the above it should be bytes[3] (and bytes[0] would give you 0x0A). But there’s a lot more data before that 0A5E 4104, which you need to take into account for that index too.

2 Likes
15

So, maybe you need to lern some Javascript first: https://www.w3schools.com/js/ or https://js.do/

1 Like
16

Hey sorry, you are right! The sensor is the CMi4110 by elvaco (https://www.elvaco.se/Image/GetDocument/en/286/cmi4110-users-manual-english.pdf). I am using the transmition mode standard and i know that the payload is long but i did not set it up that way. Its by factory.

17 
      function Decoder(bytes, port) {
  return {
    // Interpret hexadecimal 0x0230 as decimal 230, not decimal 560
    Energieverbrauch: 
     ((bytes[4] >> 4) * 1000
      + (bytes[4] & 0x0F) *100
      + (bytes[3] >> 4) * 10 
      + (bytes[3] & 0x0F))/1000,
      
    Volumen: 
     ((bytes[11] >> 11) * 100000
      + (bytes[11] & 0x0F) * 10000
      + (bytes[10] >> 10) * 1000
      + (bytes[10] & 0x0F) * 100
      + (bytes[9] >> 9) * 10
      + (bytes[9] & 0x0F))/100,
    
  };
}

I was trying this for my payload:
000C06575800000C14223902000B2D5701000B3B2008000A5A06060A5E41040C789938187002FD170000

First part of payload will be generated correctly
(get 5758 => 58+57 =>5857 => 5,857)
Second Part is not working :frowning: I guess i am doing something wrong?!
(should be: get 223902 => 02+39+22 =>023922 => 239,22
now it generates 209.02 as value)

1 Like
18

Above, you’re “shifting” 11, 10 and 9 bits to the right, in >> 11 and all. But you should only shift half a byte to the right, hence 4 bits. Try this:

    Volumen: 
     ((bytes[11] >> 4) * 100000
      + (bytes[11] & 0x0F) * 10000
      + (bytes[10] >> 4) * 1000
      + (bytes[10] & 0x0F) * 100
      + (bytes[9] >> 4) * 10
      + (bytes[9] & 0x0F))/100,

As an aside, I could not get a definitive answer on how the values are encoded from the manual (the xxxx part in the image below). How did you get those details?

Also, the payload might not always use the prefix 0C06 for Energy, and 0C14 for Volume, and then needs a different value for the decimals, a different unit, and sometimes even supplies an additional byte for the value then? Or does one configure the device for a specific output?

Excerpt from the manual:

payload format
payload format1284×634 88.9 KB

And, oops, 0C06xxxxxxxx is listed twice in that manual? :thinking:

1 Like
19

Aw perfect. No i understand what the 4 is makeing :wink: It Works. Thanks!!!

Well those xxxx are the values i do get from the sensor.
For Example my payload
000C06575800000C14[…] = 000C6xxxxxxxx0C14[…]

How i know that the 5758000 should be decoded to 5,857?
Well we can go and watch at the installed heatcounter at the Display where the sensor is installed and find out how it should be decoded in real :wink: So i can compare the real value and the payload value plus the information of the manual :wink: and thats how i know to use the unit MWh oder kWh (in this case) I hope u can understand what i mean (hard to explain in english for me - german language).

The prefixes CAN change but not in our system. We will always get those same information in the same unit and same decimals :o To include those informations in payload decoding it will be a massive rewriting of the decoder i guess?

Edit: yes our configuration gives the output in the same byte systematic (standard mode). The prefixes, units and decimals can change depending on where you install the sensor but not in our setting. So we alaway getting the same byte length, same units and same prefix :slight_smile: Lets call my solution a semi automatic decoder :smiley:

1 Like
20

Indeed, creating a more generic decoder would require quite some effort, I think. You could add some sanity checks though, like to ensure that the first byte indeed indicates the “standard” message format, and that you see the expected “DIF” and “VIF” values for the readings that you want to extract.

Just for future reference:

Meanwhile I found that “BCD” refers to binary-coded decimal, which for M-Bus indeed uses half-bytes (nibbles; 4 bits) for each digit. And things like “BCD8” then refer to 8 digit values, which need 8 nibbles, hence 4 bytes. (So, for each digit in a BCD value you’ll only see the hexadecimal values of 0x0 thru 0x9; 0xA thru 0xF are never used, which is a slight waste of bandwidth. But well, you cannot change that.)

So, you’re right about how to get the actual values.

I could not quickly find a pure JavaScript library for some generic M-Bus decoding. Reading a bit more on the M-Bus/Meter-Bus protocol, it seems that generic decoding still needs a hardcoded decision tree for expected “VIF” values (whereas “DIF” values can be used to determine the length and type regardless the VIF, but it seems many possible DIF values are never used in this very device).

Even when limiting to this very device, it seems that small errors in the Elvaco documentation will require a lot of extra field testing. Like the length of xxxxxx does not always match the number of expected bytes: the excerpt of the documentation above claims Energy is always 6 bytes and BCD8, but nevertheless shows 7 bytes in, e.g., 0CFB00xxxxxxxx. So, that should either read 0CFB00xxxxxx which would then be BCD6, or Energy could be 8 bytes in some cases? Also 0C06xxxxxxxx is listed twice in the same short excerpt, and Power is documented as “BCD8” which should probably read “BCD6”. That’s already 3 errors while quickly browsing one page in the specification.

Finally, a generic decoder would need to include the unit in its output (like apparently Energy can be one of MWh, kWh, MJ or GJ) which then needs to be respected during further processing, or the decoder would need to convert it to a single unit.