[solved] Broken communication between gateways and application

Hello,

We have an TTN Indoor and Oudoor gateway installed and our test devices are received by both of them.

In the last 4 hours we have not been receiving the hourly data from our devices via the Integration methods (we use Tago and HTTP). Last message received at 2019-09-14T15:04:30Z

Tracking this back I see that when a device sends data it is being received by TTN as both Gateways show Traffic in the TTN Console (so we can rule out the radios).

However I am no longer seeing any ‘Data’ in the Application Console. It appears that the routing between the Gateway and Application is broken.

Nothing has been changed today at all by us (devices, console, gateway or application).

Is anyone else having these types of issues? We were seeing intermittent message delivery yesterday as well but cannot confirm if this was a similar problem.

Many thanks
Gary

1 Like

My payload decoding application connected to TTN using MQTT stopped to recieve sensor data from TTN today 17:00 Swedish time… Is there anyone that can explain whats going on ?

The application have worked flawless for months, i did nothing today incomming data just stopped.

-Feeling hopeful!

there are some problems… see - https://status.thethings.network/ but this is never the ‘actual’ status.
I would check on the TTN slack channel if there is a integration problem at this moment.

if you report a problem here, please mention your region & integration if any tnx :sunglasses:

All our devices in the field have been offline since around 16:15 BST. This is really hurting us. I don’t really see anything about it on the status page. Any help would be very much appreciated.

1 Like

This matches what I have seen today with a bunch of devices last showing data in Cayenne integration around 16:00/16:15 depending on their Tx interval = most show last data around 16:10/16:13… 5 hrs down doesnt look good :frowning: I wonder Is this just a TTN issue or is TTI also seeing problem? Just for comparison between services (back-end or routing or integrations problem I wonder…)

Same problem here. Last transfer today, 14.9.2019 17.15h Berlin time. Can see the gateway traffic, but devices are not updated.

Eckehard

Hi,

Same for me, Broken since 3:15pm GMT. gateway trafic OK but no data in apps …
Good luck for those who are on the run !

Nico

Hi,
when using ttnmapper this evening I did not reach any gateway. Even at home , next to my gateway no packets arriving . When checked on TTN console I could see packets arriving to gateway but not visible in Application/Device view.

It looks like also many other gateways must be similarly impacted. Even sending SF12 packets from the balcony, from where I can typically reach 3-4 other gateways, I can not receive packets in TTN .
(well… actually packets must be arriving to TTN , as they are visible in gateway data but not in the Device-Data)

My gateway is connected to ttn-router-eu

My concern is that this issue is not yet listed on https://status.thethings.network/ so not sure if someone already looking into it.

Alexander

same issue here, also ttn-router-eu.

Issue started around 3PM UTC

Hi, same issue here as well; node backend stopped showing data packet, but TTN gateway is showing incoming traffic.

same issue here, also ttn-router-eu. Data from 2 Gateways arrive at TTN, ( as seen on the gateway console ) but not at the application. Last Data seen 2019-09-14T15:05:21.264747674Z with is similar to the Problems reported here.

We have same problem with 2 gateways and several end devices. Gateways are able to receive uplinks from devices but data are not shown in the applications. The problem was detected yesterday at 17:15 Berlin time zone, around 15 hours without data in integrations…

1 Like

Same here. No data since 17:15h

Thanks for your responses. Good to know it’s not just our account.

In terms of triangulation, can anyone confirm that their routing between Gateway and Application is working, maybe in a different region to EU.

I have pinged a couple of people at TTI to see if that tickles the support team into action.

Gary

same here , not data since sat. around 14:30utc

(EU Server)

no data in Denmark also, since 14/09 15:11h UTC. I can see the data arriving in the Gateways but not in the applications, in 3 different gateways.

Same behaviour for me. Gateway/Sensor in Germany, EU server and MQTT integration, last recievced data was yesterday (14.09.19) at 17:08 CEST (sensor does transmit every 10 minutes). Since then nothing. The packets are visible in the ttn console as gateway traffic, but are not visible in the ttn console in the corresponding application, so it seems routing to the applications is somehow broken.
Hopefully we can get at least some kind of ETA for a fix soon…

Me as well. Gateway appears up. Gateway receives data MQTT callback doesn’t appear to happen.

Soon 24h without a word from TTN staff, this matter is not managed very well… Is there no support/issue mail that could be used to notify the staff something is wrong…

1 Like

For EU no more ‘me too’ required. As many observed there is an issue. For people wanting to report issues, please use the TTN slack #ops channel. That is the official support channel to notify TTN.

Please keep in mind this is the community network, free to use for all. There are no SLAs in place and we can’t demand response/resolution within a certain amount of time. (I know it hurts, running 2 POCs myself on which this won’t reflect well)

1 Like