[solved] Broken communication between gateways and application

same here , not data since sat. around 14:30utc

(EU Server)

no data in Denmark also, since 14/09 15:11h UTC. I can see the data arriving in the Gateways but not in the applications, in 3 different gateways.

Same behaviour for me. Gateway/Sensor in Germany, EU server and MQTT integration, last recievced data was yesterday (14.09.19) at 17:08 CEST (sensor does transmit every 10 minutes). Since then nothing. The packets are visible in the ttn console as gateway traffic, but are not visible in the ttn console in the corresponding application, so it seems routing to the applications is somehow broken.
Hopefully we can get at least some kind of ETA for a fix soon…

Me as well. Gateway appears up. Gateway receives data MQTT callback doesn’t appear to happen.

Soon 24h without a word from TTN staff, this matter is not managed very well… Is there no support/issue mail that could be used to notify the staff something is wrong…

1 Like

For EU no more ‘me too’ required. As many observed there is an issue. For people wanting to report issues, please use the TTN slack #ops channel. That is the official support channel to notify TTN.

Please keep in mind this is the community network, free to use for all. There are no SLAs in place and we can’t demand response/resolution within a certain amount of time. (I know it hurts, running 2 POCs myself on which this won’t reflect well)

1 Like

For the last few weeks we have also seen the meshed server in Australia not forwarding messages to Applications on the EU server

Interregion communications being down is a known problem with backend V2. TTN explicitlty advices to run gateways, nodes and applications connected to the same zone.

Ok, but without the first „Me too“‘s in the forum I would still be spending a Sunday looking for issues on my nodes and would not have contacted the TTN. So it was helpful.

And of course there are no SLAs but a downtime of more than 17h might lead people to move to other LoRa solutions which is bad for the overall TTN community. At least I would have hoped for a more active communication on this issue. The status still only shows „Performance issues“.

Anyway, fingers crossed to get things fixed soon.

Best, Niels

4 Likes

Could it be a hardware / vendor specific issue on the gateways and nodes? I’m using those small ESP32 development boards with ABP activation and seeing the same outage as everyone else. Have they changed anything their end with authentication for example? Just a thought…

It could be, but is not. There is an issue in the back-end that affects all brands of gateways and nodes. This happened before and generally requires TTN OPS to restart some processes to resolve it.

Excellent news:

The issue has been resolved! Traffic between gateways and applications is being forwarded and posted to HTTP and MQTT as usual.

2 Likes

Yes confirmed as working here in Luxembourg, thanks for the update.

Karl R

Status page stills says: All systems up and running.

We needed to find out, why transmission stopped (Could be on our side too). If there is some issue at TTN, thats ok. If people need an SLA there are enough ways to get one.

Thank you for your support anyway.

I know that the TTN was working on this a short while ago. I suspect a few reboots sorted it.

Not sure where we should talk about this but we should probably have a level headed discussion about the correct way to notify TTN/TTI about serious like this in the future given the non SLA of the community version (although for critical applications we should probably be using a paid TTI solution).

Thanks for all your feedback to verify this was a system issue and not specific to us.

Gary

1 Like

I agree we should discuss constructivly how the community can help. A TTN watchtower group consisting of community people, identified, trusted and trained to increase response time and give TTI air to concentrate on other things.
From the TTN manifest:
We believe that this power should not be restricted to a few people, companies or nations. Instead this should be distributed over as many people as possible without the possibility to be taken away by anyone.

may I suggest to open a separate discussion topic for this

  • update