TTN Console not working properly all the time [console] [gateway] [not connected] [no traffic] [application] [no data]

bluejedi · October 22, 2020, 9:01am

The TTN Console is not working properly all the time

The following issues can manifest:

Gateway is shown in TTN Console as ‘not connected’ or e.g. ‘last seen 1 days ago’ while the gateway is actually connected.
The disadvantage of this is that the TTN Console gateway status is unreliable and that the gateway Traffic cannot be monitored in the console.
TTN Console shows no Data for an application and/or its devices, while the application actually does receive data from devices.
The disadvantage of this is that the data cannot be monitored in the TTN Console.
Possibly other quircks and combinations of above can occur.

This can be frustrating for TTN users and gateway owners and the issues occur unpredictably.
The Things Industries has made clear that these issues will not be fixed in current V2 of the The Things Network. All focus is on V3 of the public network that is scheduled to be rolled out later this year.

Since 2020 users have created many new but similar topics (and posts) about above issues. This is often done without first checking with the Search option if the issue is already (well) known. This results in many similar topics with almost identical questions.
This not only causes fragmentation of information, but having to answer the same questions over and over again gets really tiresome.

Use the Search option to FIRST CHECK if your problem is already (well) known and has been addressed already - instead of just creating a new topic or post about your issue without first checking.

Search examples

Status page

In case of issues it can also be useful to check The Things Network Status Page

descartes · June 18, 2020, 5:42pm

Is there anyway this can replace the banner at the top of the forum??

Or be a sticky somewhere obvious?

Or code a MS Paperclip style “I see you are worried about your gateway connection”

bluejedi · June 18, 2020, 5:45pm

RCB07 · June 18, 2020, 7:52pm

But…how to solve this issue? Anyone show a solve
Only talk about the NOC failure

descartes · June 18, 2020, 8:49pm

@RCB07 and anyone else hoping for a fix, which there won’t be:

There are multiple ways to check if your gateway(s) is/are online - the solution I use as it checks my own infrastructure as well, is to have a device near each gateway, use the HTTP integration which provides information on the gateways that heard the device, which you can then check your gateway appears in that list and send an email to an SMS gateway to your phone. This will let you know far sooner than a login in to a website.

I’d use the command line, but I’m currently on Virgin Media who block the required ports, but if you want something that just checks the gateway, then use the command line toolkit:

https://www.thethingsnetwork.org/docs/network/cli/

ttnctl gateway status eui-XXXXXXXXXX

will tell you the last seen date/time. A bit of [insert your own favourite scripting language here] and you can send an email to a SMS gateway to your phone.

pe1mew · June 19, 2020, 10:07am

I have created a flow in Node Red that queries all my gateways with ttnctl at an interval of 15 minutes. This method is delivering reliable results for me. Using memory to store the state of the gateway (active or not) I send messages to our community slack to indicate that a gateway went off line or when a gateway is back online.

Using another flow in Node Red, following the principle of a “Canary in a Coalmine”, I actively monitor the entire chain trough TTN effectively.

birkey · July 1, 2020, 10:45am

I think I have a rare condition that my TTIG has 2 problems. When it is listed as down it still forwards traffic. But most of the time it doesn’t forward traffic or allow a node to connect.

This forum and slack mentions it is a known issue but some people get their problem resolved. The post from blue Jedi makes me as the customer feel like they don’t care about people having problems and they want us to quit reporting it.

If this is a known issue then why are they still selling the TTIG?

They will say not all TTIG’s experience this issue. But how do they know if we have no method to report issues? Is a forum really a good method to report issues?

What makes this gateway different from other gateways?
When this year will it be fixed? It has already been 2 months

If someone wants to actually look my gateway it is eui-58a0cbfffe801426

—

descartes · July 1, 2020, 1:14pm

Apologies that you feel that way but you appear to be experiencing a different issue from the generic one listed above.

For clarity, right now, due to load on the community servers sharing information, the TTN console may show a gateway offline even though it is not and is passing on messages just fine. See posts above for ways to check your gateway other than using the web console.

It’s not that TheThings staff don’t care about this, it’s just that they are working very very hard on bringing us version 3 and the free service they offer us has suffered in a few areas but nothing significant that can’t be remedied by other means.

Also, please consider that as well as customers of the free service they provide, we are also our own customer service representatives - no one on this thread who has posted so far is a staff member.

Please please go to the TTIG topic and post your issue there, but please don’t reference the web console issue, it’s just a distraction.

bluejedi · July 1, 2020, 10:31pm

The purpose of this topic is to:

Make users aware of the Console issues described in the start topic.
Make sure that this information can be found with the SEARCH option.
Make clear that this is not going to be fixed in current V2 of the public TTN network.
And finally, to prevent users from reporting the same CONSOLE related issues (as described in the start topic) over and over again (often without checking if the issue is known already).

These issues are not specific to TTIG but are specific to the TTN Console (impacting gateways and nodes in general).

birkey · July 1, 2020, 11:24pm

I understand it is a free service and can’t wait to utilize it. This is the reason why I purchased a TTN branded gateway to help fund the cause. I think most makers realize you get what you pay for and expect a few bumps in the road. The problem is when you actually pay for a product the seller should have a certain level of responsibility to insure their customers the product they paid for will function as advertised.

I feel the customers that have purchased this gateway and in turn help fund the development of the service are being pushed aside and ignored.

If a new version of software fixes this issue great I am willing and ready to wait it out but keep us informed.

shaming your customers and making them weed through hundreds of posts that are misleading is not my idea of stellar customer service.

jensileinchen · July 2, 2020, 4:35am

hey gooood morning – interesting idea to ask status via ttnctl in node-red – i am doing it with the GET command via the http-request api from TTN ---- but sometimtes the api is not working and indicates that all our gateways are off – (but they are not) — can you share the node and/or flow ?

http://noc.thethingsnetwork.org:8085/api/v2/gateways/{{{query}}}

the {query} is variable is built before with the name of the monitored gateway (with a function node) – also i ask every 15 minutes and check the time difference – if we have a difference between the calls – the status is “UP” - otherwise if the time is same or empty i send “DOWN” ---- afterwards i use email and/or telegram bot to inform our local communtiy team about the breakdown ----

if someone is interested in the flow just send message …

Clipboard02

descartes · July 2, 2020, 10:05am

Mine works fine. Hundreds of other ones work fine. Yours has a specific issue and you’ve been directed to seek support for it in the TTIG area but haven’t done so, yet.

As an electronics manufacturer, I can totally assure you that it probably cost TTI to sell you a TTIG - what ever price you paid, the distributor price is insanely low.

We have been referring to the web console. We [currently] want to help you fix your hardware.

No one called you out, we just directed you to the correct part of the forum so we can help you in the right place. The community has asked that people stop reporting that the v2 web console is unreliably reporting the status of gateways as this is a known issue, a quick search of the forum or checking the status page indicates this. The volunteers answering questions don’t want to spin their wheels answering the same questions.

Please, please move on from here, post your issue in TTIG and hopefully between us all we can get you going, indeed your gateway could be operational by now.

descartes · July 2, 2020, 10:06am

@bluejedi / @kersing - with the addition of the NodeRed flow, we have at least three ways of checking for gateway online. Suggest this thread is locked but remains pinned.

arjanvanb · July 2, 2020, 10:13am

Well, the Node-RED flow above relies on the NOC, which is the main culprit. (Just saying that future readers may find errors in responses. Closing topics is not always a good idea… Almost never, I feel.)

pe1mew · July 2, 2020, 12:39pm

I will work on documenting it and share it here. please stay tuned.

Somsak · July 3, 2020, 2:43pm

I saw half of users in my country migrate to their own private servers.

pe1mew · July 3, 2020, 3:26pm

That is the price we pay for behaviour of TTN/TTI.

Jeff-UK · August 18, 2020, 2:16pm

Not sure if just my account but all (30+) TTN registered GW’s currently showing as Not Connected on Console GAteways page, with unpopulated last seen data etc. Am assuing its flakey Console again and that someone needs to go kick the server once more

CyngorGwynedd · August 18, 2020, 2:26pm

All our gateways are showing as not connected, and nothing from last seen, received or transmitted message either. Data is still coming through HTTP integration to our DB however, so not sure whats happening here.

laurens · August 28, 2020, 11:15am