Meshed-handler

I was using the meshed-router for my nodes at university since June , and it was working ok for AU915 frequency plan until this last weekend. I figured out that the OTAA Join Accept is using a different frequency from TTN frequency plan (https://www.thethingsnetwork.org/docs/lorawan/frequency-plans.html). Have anybody also detected this behavior ?

There is a problem with OTAA Joins at the moment on the Australian network. Some devices (mostly random) are affected. We are in discussion with the TTN Core Team and hope to have it resolved overnight.

Thanks, the instability has been solved !!!

Hi @Maj, hoping you can shed some light:

  • After about 4 months of operation on around midnight on 8/7/2020 I lost connectivity from a bunch of my Laird RS191 field sensors across a bunch of Dragino LG308 gateways. Turns out that the reason for this is that their Join-Accept’s started specifying the downlink frequency of 923.2 (vs 923.3). FYI I’m on the AU915 band.

  • After much troubleshooting I’ve discovered that due to a bug in the LG308 GUI, they were using the as923.thethings.meshed.com.au router (incorrectly listed as “meshed-router” in the GUI) as the server address on the gateway. So all my gateways in the field are pointing at this rather than the AU915 thethings.meshed.com.au router.

My running theory is that the as923.thethings.meshed.com.au router has been processing AU915 downlinks correctly up to midnight wed 8th July 2020.

All my gateways are in the field and I don’t have remote access, so hoping I can find a way to get 923.3 Join-Accepts from the as923.thethings.meshed.com.au router… BTW, all my gateways are registered on TTN with the AU_915_928 Frequency plan…

Do you know or can you advise how I could find out if there have been any changes in the oz meshed network that could have triggered this??

Thanks and kind regards,

Dave.

Is there an issue with meshed? I was running my devices 2 weeks ago with no issue but now I cannot get my devices to connect. my gateway connect correctly but I cannot get sensors to join. They are just sending join requests. Trying to use AS923 and have tried AS1 and AS2
Is there any status dashboard for meshed? very frustrating…

Just in case it helps, from Slack on Thursday July 9th (Amsterdam time):

@maj 2020-07-09 9:46 AM

Actually, the backend has undergone a significant update on Wed, then a partial rollback yesterday. If your gateways are using MQTT then for now there should be no difference, but the UDP bridges have now been replaced by the v3 GS Offloader.

They still should show the correct frequency plan though.

There’s is more about that on Slack; see also How do I get a Slack invite?

@arjanvanb Thanks Arjan for the feedback - I’ve joined the Slack channel, and reading through the thread it does look like the timing of the changes on the meshed router backend correlates with the impact to my traffic. Interestingly, my backend also reports correctly via “ttnctl gateways list” as AU_915_928, but when doing "ttnctl gateways status " shows EU_863_870.

Reported time: 2020-07-19 13:24:30 +0800 AWST
Frequency Plan: EU_863_870
Bridge: gs.v3.

[Edit][2nd Edit] Actually, if I add in “–router-id meshed-router” to the command above I get the AS_923 frequency plan:

   Reported time: 2020-07-20 00:36:09 +0800 AWST
  Frequency Plan: AS_923
          Bridge: gs.v3.
      IP Address: 1.127.104.10

So just need to figure out via @Maj if as923.thethings.meshed.com.au can still honour the AU915 frequency plan that’s assigned in TTN and listed with “ttnctl gateways list” command like it was before rather than what I assume is the default plan for the gateway itself…

Hi @davo30au and @dbrosy, I replied to you yesterday but I can’t see my posts on this thread, so trying again…

In addition to @arjanvanb’s reply,
AU915 UDP gateways should point to au915.thethings.meshed.com.au
AS923 UDP gateways should point to as923.thethings.meshed.com.au
AU915 and AS923 MQTT (and other) gateways should point to thethings.meshed.com.au (or router.au.thethings.network)

Nothing has changed in this respect, but the recent (8-Jul) upgrade means that incorrectly configured gateways will end up with the wrong downlink frequencies, hence why joins will fail.

Running ttnctl gateways info might show the incorrect band plan for your gateway. Don’t worry about this, it is incorrect. It’s because ttnctl doesn’t support the new V3 gateway offloader completely.

2 Likes

Just to be sure: “incorrectly configured” refers to using the wrong URL, right? (Not to some mismatch in Meshed’s Console, which only then yields wrong frequencies for gateways that also use the wrong URL.)

incorrectly configured

Good point. I was referring to gateways using the wrong server address, but they need to be configured for the right band plan in the console too.

So the gateway’s frequencies in global_conf and the server URL in global_conf (or local_conf) and the Frequency Plan in the TTN Console all need to match.

1 Like

The main Semtech UDP forwarder page could do with an update to reflect getting AU915 onto an explicit address.

I created a pull request to reflect this.

1 Like

This fixed my issue. awesome thank you

Hi @Maj, I’m having similar problems to those mentioned in this thread where the gateway is being instructed to use a different download frequency band to the upload frequency band. I’m based in NZ and trying to use the EU_863_870 frequency plan. The uplink frequencies are in the 860 MHz range but the gateway is being told to use a downlink frequency of 923.3 MHz.

Running ttnctl gives:

ttnctl gateways info <gateway-eui>
      Gateway ID: <gateway-eui>
  Frequency Plan: EU_863_870
          Router: meshed-router

ttnctl gateways status <gateway-eui> --router-id meshed-router
  Frequency Plan: AU_915_928
          Bridge: gs.v3.

I’m using a Semtech UDP type gateway and I’ve tried routing to thethings.meshed.com.au, and as923.thethings.meshed.com.au, and au915.thethings.meshed.com.au but it all results in the same behaviour.

I’m reasonably new to TTN and have read the associated slack thread but have run out of ideas. Do you know of a way to change the frequency plan of the underlying gateway status to be EU_863_870?

Hi @Maj

Thanks for the concise set of links. BTW, this is the first time I have seen the need for a gateway to point to au915.thethings.meshed.com.au , I always thought it was thethings.meshed.com.au for Au915. That’s the address published on TTN web site when registering a new gateway.

Couldn’t work out why I had never seen this before and went and searched and from what I can see this is the first time “au915.thethings.meshed.com.au” has been published on this forum.

I’m wondering if some of this can be rationalised and realigned with the TTN naming convention used elsewhere around the globe by editing the DNS records for some of these domains.

Just a thought.

2 Likes

Thanks @TonySmith, the problem is that on V2, each bridge handles a different gateway type. AU915 UDP, AS923 UDP and MQTT (both AU915 and AS923) are handled by different bridges.

On the V3 architecture, one Gateway Server will handle all those gateway types, as well as Basic Station, so there will eventually be just a single endpoint - which I envisage will be thethings.meshed.com.au or router.au.thethings.network (same endpoint, different DNS entry).

So yes you’re right, these should come together soon. When they do, we’ll leave the existing DNS entries indefinitely, but consider them legacy and all new gateways can point to the primary site.

2 Likes

Heads up for those not on Slack:

@Maj 2020-07-28 02:53 AM

Hello Australia users. Please be aware that at 4:15pm AEST we will be resizing the Australia cluster to improve performance and address a couple of related issues we are seeing. If all goes well, it will take less than 15 mins. Please direct any replies to the #australia channel.

See also How do I get a Slack invite?

Oh, wait, maybe that already happened, Time in Australia now - Time.is?

Thanks @arjanvanb. Yes, upgrade has been performed now.

Hello everyone (@Maj), how are you doing?

I’m wondering if there’s any current issue with the Meshed router? I have some configuration where the Gateway is sending data to the Meshed Router (udp://thethings.meshed.com.au:1700) and the Application is based on the Asia-SE Handler (ttn-handler-asia-se).

The behaviour is that I can see messages arriving in the Gateway “Data” tab, but nothing arrives in the Application “Data” tab.

ps.: I can’t migrate the Application to the Meshed handler because this is using an HTTP Integration, which is not available in the Meshed handler.

edit1: I have tried on a test gateway to use another router and that works fine. So it seems to be an issue for messages coming from Meshed gateway to be delivered to Asia-SE handler.

Thanks!

Hi @wisen, it’s the cross-region routing issue that we seem to experience on and off. (worldwide issue)

You can add any integration you like, but you need to use the local console to do it. https://console.thethings.meshed.com.au/

Thanks for the update and local console URL. I’ll check how complex can be to migrate the application the Meshed handler.

Cheers!