What to do when you get to 1000+ end nodes on a gateway

Hi,

I’m building a community safety (GPS Tracker / Emergency button) application. I have a gateway high up on a hill over looking the whole city of 120,000 people. I then have 2 other gateways closer to malls and parks where people will use the device mostly.

I’m concerned that if I got to say 1000+ or even 8000+ devices that the gateway on the hill would become obsolete as there would be too many transmissions it would see.

How do you build an IOT network for a city, to say support 30,000 people / assets?

Many thanks in advance

It’s tempting to say you worry about it when it happens.

Though that’s probably not safe.

Really, there’s not a whole lot that you as a gateway operator can do - deploying additional gateways is useful, but not sufficient.

Rather, what capacity really comes down to is having proper node firmware. Specifically, it needs to:

  • be reasonable about transmission rate
  • use ADR to speed up transmission and reduce power when near a gateway, to avoid capturing the air capacity more broadly than needed - in theory, the long range modes can be left to nodes not near a gateway, and the short range modes re-used in the neighbourhood of each gateway
  • autonomously reduce transmission rate when falling back to a slower spreading factor

A key problem is that nodes deployed when a technology is immature are likely to get at least one of these things wrong - in the short term when the airwaves are uncrowded, it works. But if people aren’t thinking about what would be needed to work in a more crowded situation, then an installed base can be created of things operating in unsustainable ways.

And once things are in the field, they rarely get improved…

How often do your devices transmit?

Currently hourly when idle and every 5 minutes when movement is detected.
I have built my system and end nodes in such a way that I can transmit downlink commands to update this when more devices are out there; the idea is to have 3hrly and 15 minutes, then keep upping these timers as required. The main thing is to keep the airwaves clear for an emergency button press, as once pressed will send an update every minute for 60 minutes or until alarm cleared.

ADR is used at all times; however on emergency button press, a single 1 byte packet is sent at SF12 advising there is an alarm for the device, the device then falls back to ADR for location updates.

I guess, continuing as is and see what happens and tune timers is the only thing I can do for now, and add gateways where we see large clusters of users. Problem is, the gateway on the hill that over looks the whole city, picks up SF7 from most locations, so even having small cell neighborhood gateways will likely contribute to the hill gateway becoming too congested.

Is there any monitoring I can put in place now to graph issues? should I be looking at the SNR vs RSSI? what’s the best way to gauge congestion etc so I can start it now and have something to compare against when issues arise.

Perhaps a downlink could be sent to ack the alarm and it could stop clogging the airwaves unless the device is on the move.

If these are trackers, I’m wondering if they will move around so much that they lose contact until they auto-adjust - which makes them questionable. I’d also wonder about OTAA vs ABP.

Get a spreadsheet out and do the math on transmission durations, how often they transmit etc. I’m not sure how the local gateways would congest the hill gateway unless they are transmitting far more than legal.

Yes and yes. There’s plenty of data to collect for free from various logs & data feeds and hard disk is cheap. Even something as simple as number of uplinks per hour per gateway will tell you where the hot spots are.

We have this option, but need for the owner to be able to know the latest location of the user; would be handy if we could detect that they moved more than 50m or something, but will have to look into that.

I too was unsure, we using OTAA at the moment; my knowledge of ABP is its more insecure, doesn’t require a join, etc. I thought ABP would be better but joins around the city have had no issues (at least while there is no congestion). What would your advice be?

Sorry, not the gateways causing more issues, but the fact that end nodes are heard by the hill even on SF7, so the hill will hear 75% of the city’s traffic even if we had gateways closer to the users.

Do I need to SNMP each gateway, or does TTN have a method of gathering this information directly via API? I should probably look, but maybe you can point me to something you know of if you don’t mind.

As I said, “unless the device is on the move”. But presumably the owner knows where they are?

It’s in every MQTT or HTTP integration packet.

For a safety / emergency system you should have detailed knowledge of the systems you use - so I’d urge you to read the TTN documentation.

Devices will be held by kids or elderly; owners assign contacts that will receive a push notification to their phone when alarm activated, parent will start running/driving to the location of the device and would need to know the most recent update, encase kid is running away from someone, or has been put in a car and taken without consent etc. Currently the device has ability to detect movement, but if its in their pocket or on a lanyard around the neck, just any movement would trigger “movement” unless it was set down on a surface. I’ll check with the suppliers and see if they can record GPS location in memory and when alarm activated, only report back location if there is movement of 50m or more; so thanks for the idea :slight_smile:

Yes we receive all these packets for our Apps (and store all the payload data including gateway rssi, Snr data rate, etc, but the gateways do not send packets themselves; so sure I can see what I’m doing myself but not if others are also using the gateways and using additional resources, hence I wondered if there was a means to monitor all gateway info other than via SNMP direct to the gateway nodes (which are often behind NAT. I will investigate this further and see if I can pull any data from the TTN API which I haven’t touched yet.

I just found the following, which I will use http://noc.thethingsnetwork.org:8085/api/v2/gateways/GATEWAYID

Realistically, to do this right, you’re going to need network capabilities for active device management beyond LoRaWAN class-A, and probably beyond LoRaWAN at all.

You’d end up with something that looked quite a bit more like a mobile (phone) network in terms of how it managed devices and the occasions, modes, and signal strength with which they transmitted.

Which raises the question if you shouldn’t just be using that, since the capability you want has been built into it as a pubic safety requirement for a few device generations now.

You are probably right, however the idea here is once off for the device and no monthly ongoing to the end user. Setting up a mobile network or using an existing one comes at cost.

I make sure people are aware that this community project should only be to compliment (additional layer) existing emergency services such as calling the police etc from a mobile phone.

The people using these devices are low social economic and don’t have access to phones due to budget etc. so having something is better than nothing in this case

If you implement this then please do not do this using the TTN community network which has no service levels and might be down for multiple days.
TTI staff have repeatedly told us the community network should not be used for mission critical applications. Emergency buttons will be critical in some situations.

Actually using an unreliable technology based on ISM frequencies is asking for trouble.

And for your sake, make sure you have a legally binding user agreement that clearly states the service can not be relied on for emergencies (although that might not be acceptable to the law if you sell them as emergency buttons) otherwise you’ll be sued sooner or later.

I held off from saying what @kersing said about safety critical. But it also appears you don’t have full control over the hardware - implementing a “have I moved as far as the GPS is concerned” is quite trivial - and 50m is the difference between hitting the button in the street and being locked in a coal cellar. So again, I strongly recommend having a very detailed understanding of all aspects of the technology so you can test it to the hilt.

And then run a competition with some local high school students for them to find ways to “break it”.

Apart from the questionable use of TTN, you can see the traffic on each gateway - other peoples SNR & RSSI isn’t particularly relevant but it will tell you how much traffic is flowing through it.

Overall, the idea is sound & worthy as long as people use the system the way you think they should. The only way to find out reality is to run a small trial along with some testing and get feedback.

Personally, if I was creating such a system, I’d use LoRaWAN and a GSM module so I have a fall back on either channel - but then as you said, it has a subscription cost - but then if something is worth protecting, a cheap $5 SIM is a small price to pay.

@icepicknz also be aware that without assisted GPS, or additonal RF references e.g. Bluetooth beacons of WiFi sniffing, there may be random errors and limits on precision and use case. Location averaging and over sampling for sending a partially corrected result over TTN/LoRaWAN may help a little. The below shows sample just grabbed from Cayenne of plot from a Dragino LGT92 from @edwin & friends (may be a good starting point for you to look at). Currently unit is in a conservatory being recharged for next outing so not quite an unblocked, clear view of open sky but typical of where your devices may find themslevs, if not quite in a coal cellar!

Note wide range of random plot locations (more outliers off picture as you can tell from the track lines) and this from a stationary GPS tracker with no assist or error correction. Note also the matrix of plots reflects quantisation and rounding errors due to truncation of the LLP Payload for Lat & Lon used by Cayenne in this case. As you can see without moving already seeing range >>50m!

image

That is an excellent example of GPS wandering about when it’s not.

Ideally, an IMU coupled with GPS averaging would eliminate much of the noise - only if the IMU reports a move - and many of the chipsets are very low current so suitable to use as an interrupt source should the GPS be polled and then a transmission made if movement outside a defined radius has been made. This can be tuned using the little known technique of testing and perhaps updated via downlink if need be, if the OP has access to the hardware.

WiFi & BlueTooth id’s would certainly be useful to locally record for lost / kidnapped as it would give those with appropriate scanners additional information on location once they get in to the area - particularly SNR/RSSI on the Wifi & BT for triangulation.

Also being able to trigger a remote buzzer like we do with High Altitude Balloons lost in wheat fields.

That might with a lot of care work on an object, but it won’t work on a person going about normal life activities (desk, bathroom, kitchen…) An IMU can’t detect travel but only accelerations. A non-radio based phone exercise distance tracker is just operating as a pedometer - counting steps events and using a pre-programmed step size.

many of the chipsets are very low current so suitable to use as an interrupt source should the GPS be polled

GPS, especially unprompted by external information, can’t just be “polled” - resolving a location takes signal observation time, and the longer its been since it was last done, the more time it takes.

I only said movement - and this scheme works for me but does require a fair amount of windowing when it’s on people/horses/etc - travel on living being looks like a lot of continuous jagged accelerations - rather like a pedometer as you suggested - one day I may get to AI.

Excepting a decent chipset’s hot fix can be the matter of seconds and for these applications it’s very rarely out of hot fix mode - you do have to accommodate enough time to download the ephemeris every few hours which you can do with a sliding window of activity and download the almanac whilst charging if you haven’t already had enough pieced together from ephemeris downloads.

Piece the two together and you can tell if you are indoors and just give up on a decent fix and wait for a longer period of movement and then see what the GPS can hear. This is why describing use cases / situations and testing stuff can yield cost effective / proportional results.

Hi @icepicknz, you could approach the problem as mobile operators do: when a GSM omni basestation starts attracting too much traffic, it is usually segmented in 3 pie slices of 120 degrees using directional antennas. That usually cuts the traffic load in 3 parts as well.

1 Like

Please see: Emergency Use of The Things Network Which Abuses Fair Access Policy

Should give you an idea about the limitations of using TTN for your proposed application.