TTN MQTT integration is not working

the OP of this thread (not me) had an issue from Sept 2nd, which is when we were affected so there’s at least a couple of us, but probably more.

Important clarification: I don’t mean to suggest the TTN MQTT broker is ‘borked’ for everyone… our issue is limited to a single application (unfortunately our main one) and most people must be unaffected.

We’ve been using LoraWAN for 6 years and TTN for a couple of years - there have been very few issues. The basic idea of receiving the data via an HTTP POST or MQTT subscription is something we’ve implemented many times over, from many different clients.

The simplest test is via a ‘mosquitto_sub’ connection to TTN, and I must have done that successfully hundreds of times by now. From ~Sept 2nd one of our TTN applications stopped accepting new subscriptions to MQTT messages, and the TTN web console shows no messages coming from those sensors either, BUT existing clients with MQTT subscriptions to that application are continuing to receive the messages. If I restart a linux server with a successfully working Mosquitto bridge connection to my TTN application, that machine will then permanently fail to reconnect.

I’m fairly confident it’s the ‘TTN Application’ that’s failing in some way (it would be hard to explain the data not appearing in the console otherwise). But I’m assuming it’s only a few of us affected so we’ll need to move on with our own failover plan to a new TTN Application.

But genuine thanks for the alternative connection tests that have been offered.

Once upon a time, in a business far-far-away, I had 550 customers all using a booking system just fine.

Except for one. Who worked at a company with networking specialists looking after the network. After some under-the-hood diagnosis, we figured out that the company firewall was messing with my load-balancer.

Maybe something is jamming up both MQTT and XHR?

So I would urge you to try someone else network at the very least as it may well uncover something peculiar about the network you are running on that no one has considered an issue.

thanks - given lockdown we have clients connecting from multiple home locations (which were ok before).

I do think we have a common access path via Mosquitto (because we’re mainly using Mosquitto bridges to connect to TTN and also testing with mosquitto_sub), and I’ve tried forcing the ‘mosquitto_sub’ to a given MQTT version, (i.e. -V mqttv31) but no joy. But if a Mosquitto/TTN incompatibility had crept in e.g. as a result of an upgrade and all our clients were up-to-date, that would cause a problem to span our multiple production and testing clients. But I think a lot more people than us would have noticed that.

However, the simplest symptom to report is “no sensor data messages for a given TTN Application displayed in the TTN web console, but being received ok on MQTT clients connected before Sept 2nd”

Is there anything in the data stream that stops someone trying?

nothing at all… so we’ve tried connecting from multiple clients on multiple networks including people’s homes.

See my “simplest symptom” comment above. To me that suggests a broken TTN Application. I don’t mean for every application on TTN, just that one.

From today the TTN MQTT is not working in EU.
Yesterday it working and received data, but today no connection

I am receiving uplinks.

Can you tell us what feedback / error message it is that tells you that MQTT is down?

Then we have something to work on.

I do not seen any error message. I try to get data from node-red and a standalone MQTT Exporer app. Both are not responding. In the console i received data,so thats is not the issue

When you say MQTT Explorer is not responding, how does that look, how can you tell?

MQTT Explorer has made a connection and shown only eu.thethings.network message. No topics ect.

Why not try one of the public test feeds, like mqtt.eclipse.org

And perhaps explore a little more - certainly validate your authentication & topics and that their uplinks - the more detail the more likely we will find the problem.

Mqtt eclipe.org is working, so that is not the problem. Nothing is changed in my application. So i try to investigate further and hopefully i find this problem.
Thanks for your help

I found a message:
12 Nov 16:21:16 - [info] [mqtt-broker:fb2a3684.cf4c38] Connection failed to broker: mqtt://eu.thethings.network:1883

It’s still working here on Virgin Media in the UK …

I just experienced half an hour outage of MQTT service in eu region. Is this a common issue with MQTT integration. Are other method like webhook more reliant?