LTAP LR8 LTE gateway : multiple network servers: routing

I run a Mikrotik LTAP LR8 in The Things Stack community V3. Created an app, added a device and that all went flawlessly, join requests, join ack, upload , all well.

Then I set up a local (192.168.1.190) Chirpstack stack, added an application and a device, switched the network server to 192.168.1.190, and that went as easy as TTN.

Now LTAP allows packet forwarding to multiple network servers. So I added both in the gateway.

My gateway forwards to both Chirpstack, status, joins, uploads, downloads, and TTN , all fine.

I created 2 (different) nodes, one registered as device in Chirpstack, the other in TTN. That works fine so far. This would allow for opening “closed Chirpstack network assigned” gateways to provide coverage for TTN.

Simultaneously registering a third node as device in both NMS failed, in practice it may not happen, and may have unintended consequences.

My questions :

  1. is it ok to run a gateway in this configuration?
  2. is anyone running the gateway like this and are my presumptions correct?
  3. did you encounter problems or are there things you definitely not do in this setup?

Thank you,

No.

A gateway can only be properly managed by a single server infrastructure.

If you try to connect it to two, it may appear to work, but two problems arrise

First, if both command it to transmit at overlapping times, at least one message will not be sent. But if the messages are not the same in both frequency and spreading factor, and there was another gateway in range that could have been used, not transmitting the assigned message due to conflict becomes a denial of service attack on the network whose transmit command was ignored.

Next, in areas where it is applicable a network server cannot track duty cycle if it is not the only thing commanding transmissions. And it has to be the network server that tracks that, because a gateway refusing because it (but not the server) knows it is out of duty cycle is also a denial of service attack.

Yes, that makes sense. What would be the use case of setting multiple NS ?

image

I am looking for a way to allow “poor man’s” peering at the gateway level.

The background is that I have limited access to a high profile location, and I can only install one gateway/antenna at that place.

I definitely want to include TTN.

I’ll dive in peering via Packet Broker…

Unfortunately that doesn’t solve schedule conflict either.

Such a scheme should as one of its primary purposes, but that one currently doesn’t - there’s no “oops, busy, try another gateway” mechanism there to prevent TTN downlinks from getting dropped on gateways that won’t send them.

Interesting.
The subject matter is fairly new to me, and complicated to say the least. So please bear with me a bit.

So what about following scenario:
A Gateway shall be controlled by ONE network server.
Two devices shall send a simultanious uplink on different frequencies to that gateway. Both of these uplinks will be received and processed. So far so good.

Now let’s assume that both uplinks are confirmed uplinks, requiring a downlink response at the same time. If I understand it right, a gateway can only send one downlink at a time . As a result one of the downlinks would get dropped (unless it can be shifted into the RX2 window?) .

Having said above, I don’t see a huge difference if above mentioned conflicts arise within a single network server setup, or as an interaction of two network servers. In other words: Once too many required downlink slots are overlapping the result of dropped downlinks would occur in a gateway, regardless if it serves multiple network servers or just one.

The issue of a network server being able to keep track of allowed duty cycle downlink time would still be an issue of course.

…I guess that refers to the Semtech UDP forwarder? Same true for LoRa Basics Station ?

Thanks
hb

LoRaWan servers often have the same uplink packet reported in by multiple gateways, and chose the best one to transmit any needed downlink.

Gateway “sharing” thus becomes a denial of service attack if that downlink is not transmitted, because the server has no way to know that the best positioned gateway is busy and it should have given the transmit job to the gateway with the second best signal instead.

None of the gateway connection schemes currently in use have an availability negotiation scheme able to actually support gateway sharing.

As an aside, you really must not use confirmed uplink on TTN - you’d burn through your downlink allowance in no time. But there are other key sorts of downlinks, especially autogenerated network management ones, which really need to actually transmit when commanded by the server.

The network server would know about both downlinks and is able to schedule them in RX1 and RX2 or assign one to another gateway (if available). If a gateway serves multiple network servers there is no central place where all transmissions are known and no way to resolve such conflicts.
The gateway will not shift anything to another window, it can’t because it doesn’t have knowledge of the windows and the transmission parameters to be used for them.

Thanks for the clarifications.

So in a nutshell that means that if one runs a private gateway with Chirpstack (or any other network server than TTN/TTI), then that private gateway is “lost” for the TTN community. In other words, the clear and loud message reads: “PLEASE DO NOT SHARE PRIVATE GATEWAYS WITH TTN/TTI”

Now I 've seen various posts by private gateway operators, which were willing to share their gateway with the TTN community. Many gateways do also allow the entry of multiple network servers.
And Chirpstack also has a UDP forwarder, which I’m sure would be causing the same problems we discussed here.

On the other hand I’ve not seen (or overseen?) clear and obvious posted advise for private gateway operators to NOT share their gateways with TTN.
Wouldn’t it make sense to add such warnings clearly visible to the TTN webpages which outline setting up gateways? I wonder how many well intending gateway operators are unknowingly sharing their private gateways with TTN , thus causing various problems?

There is a perfectly good scheme for people to share their private gateways with the TTN community that was developed (ie funded) and setup by TTI. So no one is saying do not share. If gateway manufacturers have misguidedly put some extra facilities that trashes the LoRaWAN spec, that’s down to them.

See https://www.packetbroker.net

2 Likes

@descartes @kersing I need to think more on this when not just dipping in an out of Forum and distracted by day jobs but whilst PB potentially solves the problem of traffic routing and sharing between multiple network, or simpy bridging between networks as in V2 to V3 TTN traffic, I dont think it helps solves the problem where NS1, NS2, …NSn, TTNNS(V2 or V3), TTINS are all working through PB and using a given GW - there is no way for NS2(say) to know that NS4 is scheduling a downlink at the same time through the same GW and there is still a risk of collision, nor any way to know what NS2’s view of used duty cycle at that specific GW vs NS4’s view, right? (I need to look deeper as I say).

Nope, just a few allow it. Most gateways run software from Semtech (UDP packet forwarder of BasicStation) which doesn’t allow multiple backends.

I am sharing >15 private gateways with the TTN community. As in I privately paid for those gateways and connected them to TTN to allow anyone (within TTN) to use them.
I do not see the need to run a private LoRaWAN backend as my use cases all fall within the TTN fair access policy and sharing the gateways with the community is important to me.
Looking at the logs of my gateways I notice local (NetID 00) deployments that couldn’t use TTN and are even breaching legal limits. If there are too many of those LoRaWAN will become unusable (we share the same radio spectrum after all) so everyone looses.

From what I understood PB should facilitate this. But it might be better to ask the appropriate authority on the subject. @johan, can you comment?

1 Like

So we have

  • one church tower,
  • one gw/antenna allowed by the building owner.
  • and both a private and TTN community competitor for the location.

The Private Operator is open to forward whatever traffic as long as it does not hinder them, or cost them extra money or difficult arrangements or configuration.

The TTN community is open to forward whatever traffic as long as the GW is open for TTN.

A “how to” on what is the preferred way of handling this type of situation would be welcome.

The worst outcome would be both parties negotiating with the church owner, leading to 2 gateways: one TTN and one Private.

Further: apart from all the previously mentioned issues, am I right that it makes no real difference where multiple-server-packet-forwarding takes place, either on the gateway or by a dedicated multiplexer? ( eg by How does the ChirpStack Packet Multiplexer work? - Gateways - ChirpStack Community Forum )

Ill try to reach out to MikroTik to ask them what the use case is for this setting.

2 Likes

I have to be heavily medicated before people can cope with my advanced negotiation skills so I’m entirely sure I’m the right person to ask. Or is it they have to be medicated. Can’t remember.

This isn’t really a technical problem, more about what the community would like or benefit from coupled with appealing to the private operators sense of Corporate Social Responsibility & any associated PR benefits for them.

If the private operator can meet the non-trivial requirements to link to the Packet Broker service, I’d ask them to do that, promise to put their name as a Platinum sponsor on the community website and put the gateway & antenna you have somewhere else.

:laughing: Good suggestion to improve your negotiation skills !

From what I understand, after re watching the 2020 TTN conference video on YouTube Link about packet broker is that you need to have an assigned NetID which is only available as a member of the LoraWAN alliance. This costs ~US$3,000 per annum.

This is out of reach of most people who want to have the ability to operate a private gateway (or cluster of) to achieve a private LoRaWAn capability but provide the community at large the benefit of the gateway participating with TTN and passing (up and down link) traffic to users within the coverage of their gateway (or cluster of).

Please correct me if I am wrong.
Perhaps packetbroker needs to consider some sort of networking RFC1918 (192.168.x.x 172.16.x.x 10.x.x.x) equivalent for the NetID [edit… am aware of " NetID 00"] where localised use for the private is possible as a mask and the catch-all traffic is sent on.

Anyone doing this seriously is likely to be operating in a commercial context, so the additional costs shouldn’t be much in the grand scheme of things. You can rent a small block of IDs from TTI that will cost less than membership.

The alternative is to run the private network using a TTI instance that can then automagically peer.

It’s not clear what your project is about or indeed how many gateways make a cluster - why do you need a private network?

That is mistaken

While packet broker should have been designed to solve this problem, specific questions here in the past reveal that at present it actually doesn’t do anything to address it at all.

How so?

Why would TTI have put so much effort in to creating it if it wasn’t going to allow networks to peer?

Who knows.

Just like feeding multiple network servers directly, it makes it look like peering works, but actually the networks denial of service attack each other because theres no logic to let the loser of a conflict try a different gateway.