Mosquitto downlink messages do not come through my RAK2287 gateway

TTN Network and Services The Things Stack v3
1

Hi all,
I am trying to get a standard Mosquitto downlink message (from my PC) through my RAK2287 gateway(EU868) to my node (Heltec V2 with THP sensor and two LEDs).
What does work ok:

  • my node sends the THP data to the gateway every 2 minutes. I see the data coming to the gateway, and in the TTN Console.
  • When I do a simulated downlink message in the console to the node, the message comes through and the data in the node (send to the serial interface on Arduino IDE) is correct.
  • When I do a:

mosquitto_sub -v -h eu1.cloud.thethings.network -t “v3/bme280-sensor-xxxx@ttn/devices/eui-70b3d57ed005c2xx/up” -u “bme280-sensor-xxxx@ttn” -P “NNSXS.4EDQZHGKO7IIFCRNGM2RLNH26L6W5KVO6GMFFPQ.7NXP5CXF5KMOWJCC7W54XNVBZNLZ3P5G4DEZD6Z3KVDYB7URLWxx” -d (changed a few characters to x’s)

I get

Client null sending CONNECT
Client null received CONNACK (0)
Client null sending SUBSCRIBE (Mid: 1, Topic: v3/bme280-sensor-xxxx@ttn/devices/eui-70b3d57ed005c2xx/up, QoS: 0, Options: 0x00)
Client null received SUBACK
Subscribed (mid: 1): 0

And nicely every two minutes a message from TTN containing about all details of my node. Good!

  • When I do a:

mosquitto_pub -h eu1.cloud.thethings.network -t “v3/bme280-sensor-xxxx@ttn/devices/eui-70b3d57ed005c2xx/down/push” -u “bme280-sensor-xxxx@ttn” -P “NNSXS.FOI3MH7XQWJXSQDESNB3IL7UZJONE662NKWSTHY.IEUPZTPCSVVH7ZO5YFLAYFBS6JBIUUC7A4ZDGHIGISUNXUWFR7xx” -m “{“downlinks”:[{“f_port”: 15,“frm_payload”:“vu8=”,“priority”: “NORMAL”}]}” -d

I get

Client null sending CONNECT
Client null received CONNACK (0)
Client null sending PUBLISH (d0, q0, r0, m1, ‘v3/bme280-sensor-xxxx@ttn/devices/eui-70b3d57ed005c2xx/down/push’, … (59 bytes))
Client null sending DISCONNECT

BUT:

It is radio silence,…no message in the TTN console, no message in the gateway and no message in the node.
I did play with types of messages (hex, string, etc.) which brought me no further.
Also with names, id’s, API keys, and read everything I can get my hands on.
It goes over my head for some reason,…
Any help, docu links, or hints, would be highly appreciated.

Loek

2

This would be in breach of the Fair Use Policy - please go and read it - it’s unlikely the temperature & humidity will change that fast so it’s a rather non-sensical rate.

Unlisted topic pending confirmation of FUP

4

Hi Nick,
Thanks for your response. Yes, I realize that every two minutes is to much , but I try to minimize it as much as possible.
My problem is that in order to get a message down to my node I have to first send one up. And that slows down the test process enormously. So I have to try to test as fast as possible while not breaching the FUP. Anyway, sorry that I did! I will slow down my testing according to the FUP.
Has my problem something to do with my breaching, or is there another reason that my Mosquitto messages are not coming through ?
Hope you can help me,
Loek

5

If you are clear there is a FUP, you’ll also be super-clear that there is a 10 downlink limit as well.

It’s not a problem as you are unlikely to be able to do anything meaningful in the 2 minutes between uplinks. So it could be every 10 minutes or even better, on-demands, see below.

And your first goal has nothing to do with uplinks - you just need to get it on to the console.

And if you’ve got it on the console, it’s in the system and you’ve shown that once it’s in the system that the downlink gets delivered.

So you’re problem has nothing to do with device or the gateway which means it doesn’t need a constant stream of uplinks.

If / when you do get to a point where you need to prove to yourself that the downlink work for real, then have a pushbutton on the device that triggers an uplink. You can then review your results without the console & serial log becoming clogged up with a constant barrage of uplinks.

TL;DR: There is no need to use uplinks to do this.

Important note: Community consensus is that downlinks aren’t an every day thing - LoRaWAN is not a command & control system - downlinks are for changing settings for autonomous local decision making - so if your design requires many downlinks, more than one a fortnight, then you may need to review your design.

7

So, looking at your actual code, some formatting for clarity:

mosquitto_pub 
-h eu1.cloud.thethings.network 
-t “v3/bme280-sensor-xxxx@ttn/devices/eui-70b3d57ed005c2xx/down/push” 
-u “bme280-sensor-xxxx@ttn” 
-P “NNSXS.FOI3MH7XQWJXSQDESNB3IL7UZJONE662NKWSTHY.IEUPZTPCSVVH7ZO5YFLAYFBS6JBIUUC7A4ZDGHIGISUNXUWFR7xx” 
-m “{“downlinks”:[{“f_port”: 15,“frm_payload”:“vu8=”,“priority”: “NORMAL”}]}” 
-d

And now, without “smart-quotes”

mosquitto_pub 
-h eu1.cloud.thethings.network
-t "v3/bme280-sensor-xxxx@ttn/devices/eui-70b3d57ed005c2xx/down/push"
-u "bme280-sensor-xxxx@ttn" 
-P "NNSXS.FOI3MH7XQWJXSQDESNB3IL7UZJONE662NKWSTHY.WFR7xx" 
-m "{"downlinks":[{"f_port": 15,"frm_payload":"vu8=","priority": "NORMAL"}]}" 
-d

Still not correct - spot the un-escaped " hidden in the mix!

And from Eclipse Mosquitto | The Things Stack for LoRaWAN

mosquitto_pub \
-h "thethings.example.com" \
-p "1883" \
-u "app1" \
-P "NNSXS.VEEBURF3KR77ZR.." \
 -t "v3/app1/devices/dev1/down/push"  \
-m '{"downlinks":[{"f_port": 15,"frm_payload":"vu8=","priority": "NORMAL"}]}'

Smart quotes may be a CopyNPasta issue, or it may be in your original command line, either way, you need to use a text only editor to prepare or copy commands around. I don’t do Windows so I don’t know about using \ as a line ending to split things over several lines, but it really helps having one option per line.

Your user name looks odd - using a device description for an application name can cause some confusion even when you know that’s what it is.

But overall, you can do a compare & contrast with the almost tidied up version (2nd block) with the example from the docs (3rd block) along with the hint.

8

ok, the policy is clear now, and I did implement the ‘push > one uplink message’ button.
I also understand now that testing the downlink from Mosquitto doesn’t need an uplink message, just a node device which is alive and connected to the active gateway. Now I will dive into the code,…tnx again Nick.

9

Don’t dive - it’s a shallow end issue and you may break you neck - relax in to the compare & contrast of the formatting of quoting quotes - when you spot it, it may really ‘tick’ you off!

10

Hello again,
I noticed the difference between ', " , en the smart one as you call it. dubble curvy small characters at the top of the line. And apparently there is some mix up in UNIX/WINDOWS.
I think that you means with

  • spot the un-escaped " hidden in the mix!
    that I have a character in the block which is not visible. I can’t find it ,…

So to avoid confusion, I installed debian on my PC and also Mosquitto. I go on with debian until I understand it fully.
So on debian I have typed the complete command in the command line without any copy and paste from any environment. I issued that command and:

  • in the console/Gateways/Live data, I see no incoming message
  • in the console/Applications/node, I see
    two messages, one with “Subscribe application” type and one with “Unsubscribe application” type.
    Is that ok ?? I expect a notification of a message in the gateway, and in the application overview.
    I dont see any message content anywhere. But I have the feeling that I close,…
    The event details of the subscription message are:
    {
    “name”: “as.application.subscribe”,
    “time”: “2023-04-30T18:21:47.049491956Z”,
    “identifiers”: [
    {
    “application_ids”: {
    “application_id”: “bme280-sensor-loek”
    }
    }
    ],
    “correlation_ids”: [
    “as:subscriber:01GZ9PCGN9NMHEGJEW8JN95NCM”
    ],
    “origin”: “ip-10-100-12-177.eu-west-1.compute.internal”,
    “context”: {
    “tenant-id”: “CgN0dG4=”
    },
    “visibility”: {
    “rights”: [
    “RIGHT_APPLICATION_LINK”
    ]
    },
    “unique_id”: “01GZ9PCGN9YHE17G4TAAQJKVZB”
    }

Is that what is should be ???
Can you give me a last push to get it done ?
Loek

11

OK, not so much invisible but hard to see if you have lots of " in a command line. So you’re looking too hard, try this, yours on top, the one from docs underneath:

-m "{"downlinks":[{"f_port": 15,"frm_payload":"vu8=","priority": "NORMAL"}]}" 
-m '{"downlinks":[{"f_port": 15,"frm_payload":"vu8=","priority": "NORMAL"}]}'
12

Yes, I see the difference. On my screen I see that the -m option is surrounded by double straight quotes in my command and the single quotes in the one from docs. I noticed that before and tried already several options. I give you the outcome of it.
In the windows terminal:
surrounding the -m with double straight quotes I get the mosquitto message:
Client null sending CONNECT
Client null received CONNACK (0)
Client null sending PUBLISH (d0, q0, r0, m1, ‘v3/bme280-sensor-loek/devices
/eui-70b3d57ed005c2ad/down/push’, … (60 bytes))
Client null sending DISCONNECT
In the Gateway Live data only “Receive gateway status” messages
In the Application Live data “Subscribe application” message, followed by a
unsubscribed application" message
surrounding the -m with single straight quotes I get the error message from mosquitto:
Error: Unknown option ‘15,frm_payload:vu8=,priority:’
On the laptop in debian:
surrounding the -m with single straight quotes I get the mosquitto message:
CONNECT, CONNACK, PUBLISH, DISCONNECT again.
Nothing in the gateway, and the sub and unsubscribe message in the application Live
data
surrounding the -m with double straight quotes I get an error message from Mosquitto:
Unknown option ‘-m…’

Can you imagine that I am a bit confused,…
In the windows terminal the double quotes give me an acknowledge on the connection with ‘sub and unsubscribe application’ message in the application live data.
In the debian however I get an acknowledge with only a single quote, with the messages in the application.

To make things even easier I changed the -m in -m ‘0’ in debian.
I get an acknowledge with ‘1 byte sent’.

(I also tried to look at the effect of changing one char in the NNSN…id. Well this is consistent.
I see a ‘failed to connect’ message in the application live data)

My conclusion up to now:

  • stick to the debian environment to avoid problems with quotes
  • I can connect to the ttn server, so my user/password -u/-P combi is ok
  • the simplest message -m ‘0’ is sent by mosquitto
  • In debian, using the -m ‘xxx’ I get the sub and unsub messages in the application live data.
  • nothing in the gateway live data
  • Is the problem then in the -t part ?? I changed the double quotes in single quotes. No difference.

I am afraid I breached the 10 downlinks again, so I will be off again.

You can imagine that I am very eager to get this solved, and I am frustrated but very determined to get it done. Therefore I appreciate your determination to help me out a lot !!
Loek

13

Could you reformat your post (click on the pencil tool) and use the </> formatting for logs and put in a bit more white space - it’s hard to differentiate your comments from the cmd line entires from the cmd line output. And going forward look at the preview and figure out if others can make easy sense of it all.

Fundamentally you can’t use " to quote a string if that string has " in it. How would the command line know that you don’t mean the string has ended. The JSON spec says strings should be enclosed in " so you need to quote the JSON you are sending with a ’ - sometimes called a tick.

You have a template from the docs that I’ve quoted above - you could try substituting the app id, dev id & your API key in to it in a text editor and giving that a go. I’ve just tried that and “it works for me” on macOS command line, here’s what I used:

mosquitto_pub \
-h "eu1.cloud.thethings.network" \
-p "1883" \
-u "descartes-interval-dev@ttn" \
-P "NNSXS.IXJI...GZIR" \
-t "v3/descartes-interval-dev@ttn/devices/did-stm32wl01/down/push" \
-m '{"downlinks":[{"f_port": 15,"frm_payload":"vu8=","priority": "NORMAL"}]}'

I can confirm Windows doesn’t like multi-line commands. Nor does it seem to cope with the quoting scheme of ’ and " - and if I change the macOS cmd line version to use ’ in the JSON, nothing appears, most likely because it’s not valid JSON.

So by substituting your info I’d expect it to work on Debian. How mosquitto_pub on Windows allows a message to be set you can investigate whilst I take the wife out for lunch.

14

Your downlink messages should appear in the application data stream. Only after an uplink you might see a downlink at gateway level, however that assumes that gateway will be chosen for the downlink.

As long as you don’t see the downlink message in the application data stream there is no need for you node to uplink (and poll for a downlink). You can push a downlink 100 times to the back-end as long as your node does not uplink. (And you ‘replace’ the queued downlink using: ‘Instead of /push, you can also use /replace to replace the downlink queue.’)

15

Good spot from @kersing - as mentioned above, it’s all about the device view, the gateway doesn’t need to be in on the action.

Just had a fiddle with Windows - mosquitto_pub doesn’t appear to like the -m for JSON formatted payload - so I put the JSON in a file and used the -f option - and it worked fine.

16

Hello Jac,
Tnx for the response.
OK, I understand that I should not look for a downlink in the gateway live data, but in the application live data. And indeed I see something happening there after a mosquitto command. I see:

12:21:46 bme280-sensor-loek Unsubscribe application
12:21:46 bme280-sensor-loek Subscribe application

So my mosquitto command is doing at least something in my application. The question is, whether these two messages are correct or pointing at a error I make.
I expect something like “downlink received, data processes” or something similar. And I don’t see any reference to the content of the downlink mesage.
Can you explain the meaning of these two lines (Sub and Unsubscribe)?
Loek

17

That’s when you use mosquitto_sub …

18

I will try,…
I see that you put “did-stm32wl01” as device id in the TOPIC.
Is that the description of the device or the End Device id.
I have there eui-xxxxxxxx. Is that correct ?? Or should I also use the name ?

19

Nick,
I did it !!! (using the eui-…)
I have now

  • a Subscribe application message
  • a Receive downlink data message
  • a Unsubscribe application message

And the downlink message contains DATAAAAAAAA !!! Yes.

Next thing is to learn JSON, looks like it is very widely used for this type of applications.

Again, very much appreciated your help!!

Probably I will come back again in the future,…because you made the difference for me !

Loek

20

Erm, the template and the docs say device id, so I use the device id.

Device Id is for you to identify a device - which is a bit hard to do if you are using the EUI - whereas I actually have device id’s like “broken-pro-mini-rfm-plantpot” because it is a test DIY device that I can’t use for much other than tracking soil moisture levels.

Great - but be careful you don’t fixate on specifics - try to look at it in generic terms - it wasn’t that you used something that started with eui, it was because you used the device id as per docs. This will help with JSON …

:rofl: