New API for gateway mapping, status and info

TTN Network and Services
,
1

I’m happy to announce that we have a new API for fetching gateway locations, online status and other info for The Things Stack Community Edition, Cloud, Open Source and Enterprise: Packet Broker Mapper. Since we wanted to make gateway information available in one place, we built a new service for this in Packet Broker. With Packet Broker being the backbone for LoRaWAN traffic, making gateway information available in a central place allows for better insight in the entire ecosystem and discovery of (private) networks to exchange traffic with.

Getting started

The OpenAPI (a.k.a. Swagger) definition: https://mapper.packetbroker.net/api/v2/openapi.json (to play around with this, go to https://editor.swagger.io/, click File → Import URL).

Examples

Get all gateways in Amsterdam connected to The Things Stack Community Edition:

https://mapper.packetbroker.net/api/v2/gateways?distanceWithin%5Blatitude%5D=52.3676&distanceWithin%5Blongitude%5D=4.9041&distanceWithin%5Bdistance%5D=7500&netID=000013&tenantID=ttn

Get all gateways, globally, currently online and connected to The Things Network V2:

https://mapper.packetbroker.net/api/v2/gateways?online=true&netID=000013&tenantID=ttnv2

Get an individual gateway:

https://mapper.packetbroker.net/api/v2/gateways/netID=000013,tenantID=ttn,id=bb1st-jansmuts-1

The Things Network uses NetID 000013 and tenant ID ttn (V3) and ttnv2 (V2).

Rate limiting and performance

This API is rate limited. Do cache responses.

You can use the limit and offset query parameters to paginate through results. You can also use updatedSince query parameter to specify a timestamp since you last fetched information, so you only get new results.

Also use Accept-Encoding: gzip to reduce bandwidth. It’s faster and better for the planet.

Backwards compatibility

This replaces two existing APIs: the https://www.thethingsnetwork.org/gateway-data/ endpoint and the NOC (part of V2). The former now uses the Packet Broker Mapper API as data source, so applications using the old API now receive up-to-date information (including our very own www.thethingsnetwork.org gateway map as we work on a redesign of the homepage). Please update your API clients to consume the new API, as we will be shutting down the /gateway-data endpoint by the end of 2021. The NOC is already gone.

FAQ

Is this an alternative to TTN Mapper?

No; TTN Mapper will use Packet Broker Mapper as a data source for gateway locations. We currently don’t plan to implement coverage mapping functionality for site surveys.

How to hide my gateway location or online status?

All The Things Network gateways are public. However, you can hide your location or online status. Go to your gateway in The Things Stack Console, click General settings and uncheck the Public checkboxes under Gateway location and/or Gateway status. It takes roughly 10 minutes for changes to be effective.

How to configure gateway visibility in my The Things Stack Cloud, Enterprise or Open Source private network?

As an administrator of a network that is connected to Packet Broker, you can configure your gateway visibility and fine-tune exactly what information about your gateway you want to expose: location, antenna count, whether fine timestamps are produced, frequency plan, online status and RX/TX packet rates (per hour). The Things Stack get support for this to manage this yourself. Until then, please reach out to support (Cloud or Enterprise) or DM me (Open Source).

Happy to answer any questions that you may have!

19 Likes
2

A simple flow in node-red to get all gateways in a radius around a coordinate from The Things Stack Community Edition

image

[{"id":"b062438a.ed523","type":"tab","label":"Gatewat RX, TX","disabled":false,"info":""},{"id":"78c4d268.45fb8c","type":"http request","z":"b062438a.ed523","name":"","method":"GET","ret":"txt","paytoqs":"ignore","url":"","tls":"","persist":false,"proxy":"","authType":"","x":730,"y":240,"wires":[["b002c6ff.3b4ba8"]]},{"id":"c22ab33d.13229","type":"debug","z":"b062438a.ed523","name":"","active":false,"tosidebar":true,"console":false,"tostatus":true,"complete":"true","targetType":"full","statusVal":"payload","statusType":"auto","x":1050,"y":180,"wires":[]},{"id":"b002c6ff.3b4ba8","type":"json","z":"b062438a.ed523","name":"","property":"payload","action":"","pretty":false,"x":870,"y":240,"wires":[["862912eb.88b07"]]},{"id":"862912eb.88b07","type":"split","z":"b062438a.ed523","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":990,"y":240,"wires":[["c22ab33d.13229","99d71b81.06ea88"]]},{"id":"977590a6.7779c","type":"ui_text_input","z":"b062438a.ed523","name":"","label":"Lat","tooltip":"","group":"466c0f08.3557","order":2,"width":2,"height":1,"passthru":true,"mode":"text","delay":300,"topic":"topic","topicType":"msg","x":270,"y":120,"wires":[["bfd01747.ce8a18"]]},{"id":"a43e478c.841518","type":"ui_text_input","z":"b062438a.ed523","name":"","label":"Lon","tooltip":"","group":"466c0f08.3557","order":3,"width":2,"height":1,"passthru":true,"mode":"text","delay":300,"topic":"topic","topicType":"msg","x":270,"y":160,"wires":[["336a12f0.bbe8de"]]},{"id":"dcee690f.aa5a58","type":"ui_text_input","z":"b062438a.ed523","name":"","label":"Dist","tooltip":"","group":"466c0f08.3557","order":4,"width":2,"height":1,"passthru":true,"mode":"text","delay":300,"topic":"topic","topicType":"msg","x":390,"y":200,"wires":[["9b7ab080.166b"]]},{"id":"dafd3bd4.c78008","type":"ui_button","z":"b062438a.ed523","name":"","group":"466c0f08.3557","order":6,"width":2,"height":1,"passthru":false,"label":"get","tooltip":"","color":"","bgcolor":"","icon":"","payload":"","payloadType":"str","topic":"topic","topicType":"msg","x":390,"y":240,"wires":[["e4622917.4bad98"]]},{"id":"bfd01747.ce8a18","type":"change","z":"b062438a.ed523","name":"","rules":[{"t":"move","p":"payload","pt":"msg","to":"lat","tot":"flow"}],"action":"","property":"","from":"","to":"","reg":false,"x":450,"y":120,"wires":[[]]},{"id":"336a12f0.bbe8de","type":"change","z":"b062438a.ed523","name":"","rules":[{"t":"move","p":"payload","pt":"msg","to":"lon","tot":"flow"}],"action":"","property":"","from":"","to":"","reg":false,"x":450,"y":160,"wires":[[]]},{"id":"4d4db5c0.620fdc","type":"change","z":"b062438a.ed523","name":"","rules":[{"t":"move","p":"payload","pt":"msg","to":"dist","tot":"flow"}],"action":"","property":"","from":"","to":"","reg":false,"x":730,"y":200,"wires":[[]]},{"id":"e4622917.4bad98","type":"function","z":"b062438a.ed523","name":"","func":"let  lat = flow.get('lat');\nlet lon = flow.get('lon');\nlet dist = flow.get('dist')\n\n\nmsg.url = \"https://mapper.packetbroker.net/api/v2/gateways?distanceWithin%5Blatitude%5D=\" + lat + \"&distanceWithin%5Blongitude%5D=\" + lon + \"&distanceWithin%5Bdistance%5D=\" + dist + \"&netID=000013&tenantID=ttn\";\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":540,"y":240,"wires":[["78c4d268.45fb8c","3cb07fe3.14cdc"]]},{"id":"80ae19d3.448f98","type":"ui_worldmap","z":"b062438a.ed523","group":"466c0f08.3557","order":8,"width":0,"height":0,"name":"","lat":"","lon":"","zoom":"","layer":"","cluster":"","maxage":"","usermenu":"hide","layers":"hide","panit":"false","panlock":"false","zoomlock":"false","hiderightclick":"true","coords":"none","showgrid":"false","allowFileDrop":"false","path":"/worldmap","x":1040,"y":420,"wires":[]},{"id":"7e5bf05.187dc1","type":"worldmap","z":"b062438a.ed523","name":"","lat":"","lon":"","zoom":"","layer":"","cluster":"","maxage":"","usermenu":"show","layers":"show","panit":"false","panlock":"false","zoomlock":"false","hiderightclick":"false","coords":"none","showgrid":"false","allowFileDrop":"false","path":"/worldmap","x":1040,"y":360,"wires":[]},{"id":"e5b54b5.ee797b8","type":"http request","z":"b062438a.ed523","name":"","method":"GET","ret":"txt","paytoqs":"ignore","url":"","tls":"","persist":false,"proxy":"","authType":"","x":950,"y":300,"wires":[["fbda089e.9f1e08"]]},{"id":"f9e66a57.636ee8","type":"function","z":"b062438a.ed523","name":"","func":"let id = msg.payload.id;\nlet lat = msg.payload.location.latitude;\nlet lon = msg.payload.location.longitude\nlet rxRate = msg.payload.rxRate;\nlet txRate = msg.payload.txRate;\nlet online = msg.payload.online;\n\nif (online == true) {\n    color = \"#0000ff\";\n}else{\n    color = \"#ff0000\";\n};\n\n\nmsg.payload = { \"name\":id, \"lat\":lat, \"lon\":lon, \"rxRate\":rxRate, \"txRate\":txRate, \"online\":online, \"iconColor\":color }\n\n\n\n\n\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":760,"y":360,"wires":[["7e5bf05.187dc1","d1de4ceb.90304"]]},{"id":"d1de4ceb.90304","type":"debug","z":"b062438a.ed523","name":"","active":true,"tosidebar":true,"console":false,"tostatus":true,"complete":"true","targetType":"full","statusVal":"payload","statusType":"auto","x":750,"y":420,"wires":[]},{"id":"9b7ab080.166b","type":"function","z":"b062438a.ed523","name":"","func":"msg.payload = msg.payload * 1000;\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":550,"y":200,"wires":[["4d4db5c0.620fdc"]]},{"id":"99d71b81.06ea88","type":"function","z":"b062438a.ed523","name":"","func":"let id = msg.payload.id;\n\n\nmsg.url = \"https://mapper.packetbroker.net/api/v2/gateways/netID=000013,tenantID=ttn,id=\" + id +\"\";\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":780,"y":300,"wires":[["e5b54b5.ee797b8"]]},{"id":"fbda089e.9f1e08","type":"json","z":"b062438a.ed523","name":"","property":"payload","action":"","pretty":false,"x":1090,"y":300,"wires":[["f9e66a57.636ee8"]]},{"id":"c13553ed.99c8a","type":"ui_text","z":"b062438a.ed523","group":"466c0f08.3557","order":1,"width":0,"height":0,"name":"","label":"Coordinates to be in decimal and the distance in km","format":"","layout":"row-spread","x":250,"y":80,"wires":[]},{"id":"20417d2b.d5dac2","type":"inject","z":"b062438a.ed523","name":"","props":[{"p":"payload"}],"repeat":"","crontab":"","once":true,"onceDelay":"1","topic":"","payload":"lat","payloadType":"flow","x":120,"y":120,"wires":[["977590a6.7779c"]]},{"id":"118bc15.972113f","type":"inject","z":"b062438a.ed523","name":"","props":[{"p":"payload"}],"repeat":"","crontab":"","once":true,"onceDelay":"1","topic":"","payload":"lon","payloadType":"flow","x":120,"y":160,"wires":[["a43e478c.841518"]]},{"id":"ad4ab4f2.dcfc48","type":"inject","z":"b062438a.ed523","name":"","props":[{"p":"payload"}],"repeat":"","crontab":"","once":true,"onceDelay":"1","topic":"","payload":"dist","payloadType":"flow","x":120,"y":200,"wires":[["133addcc.e8b832"]]},{"id":"133addcc.e8b832","type":"function","z":"b062438a.ed523","name":"","func":"let dist = msg.payload / 1000;\nmsg.payload =dist;\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":260,"y":200,"wires":[["dcee690f.aa5a58"]]},{"id":"3cb07fe3.14cdc","type":"debug","z":"b062438a.ed523","name":"","active":true,"tosidebar":true,"console":false,"tostatus":true,"complete":"true","targetType":"full","statusVal":"payload","statusType":"auto","x":550,"y":300,"wires":[]},{"id":"466c0f08.3557","type":"ui_group","name":"Location","tab":"6bf50ff0.5f5d2","order":1,"disp":true,"width":30,"collapse":false},{"id":"6bf50ff0.5f5d2","type":"ui_tab","name":"TTN Map","icon":"dashboard","disabled":false,"hidden":false}]
'''
5 Likes
3

Nice. So you need node-red-contrib-web-worldmap and node-red-dashboard plugins, that you can install via the palette. Then I get an error that ui_worldmap doesn’t exist. There’s no map on the dashboard also. What’s missing?

4

You need both nodes, ‘ui_wordmap’ from the ‘dashboard plugin’ and ‘worldmap’ from the ‘location plugin’

image

image

Also check your “Web Path” is set to the same path, as this is what links your two “worldmaps”
image

The ‘ui_worldmap’ is what ‘pulls’ the map into the dashboard.
image

3 Likes
5

LOL, Johan providing support for Johan!

1 Like
6

LOL, We are taking over, so you better change you name, otherwise we will not recognize :smiley:

7

Cool, that works, thanks!

8

Please mark it as a solution :wink:

9

Not preferable as that would auto close the topic (after 24h inactivity).

10

Hi @johan

In V2 we had a counter for tola-messages uplink and downlink. In the new V3 console this number is set to 0 is you restart the gatway. Or set to 0,00 because some other reasons.

I got the URL in your sample-URL for my gateway working. But I do not see the number of uplink & downlink messages as shown in the V3 console.

I looked into the JSON file, but my knownledge is not that good.

Can the API give the number of uplink & downlink messages?

With this, you can create nice dpublic dashboards for networks like my: www.ttn0478.nl.
All gatewys in a map: GREEN and RED (offline). Ond with coloring the gateway with the most uplinks…

Hope to hear

Marc

11

A number of others have asked about this from various other APIs

It may be more expedient in the short to medium term to hold the values on the database and if you get a lower figure, you can assume there has been a restart and accumulate them

12

@descartes , do you mean the total of uplink and downlink messages will be available in the API on short/medium time? And that I store these numbers also in my own database and recalculate the total of my gateway in a month for historical data?

That is no issue if the numbers will be available in the API.

13

The JSON have “rxRate”:70.93811,“txRate”:0 they are measured every 4-6 min and are indicative of packets per hour

14

@Johan_Scheepers I do apologize, but is packages the same as a message? And if this is every 4 to 6 minutes, you do not have actual numbers to calculate with. Marc

15

I think we all refer the the same thing a a few different ways, packets / messages / uplink / downlink and most probably a few more terms that can be used.

Don’t have a exact number, I think they measure packets and the min and then simply packets/ min*60, but sure @johan can give a more accurate answer. Also seen these fields are not available if you quarry to often.

There are a few other fields I would like to know there significance, event.clientX , event.clientY and event.bbox.

16

Yep, Gateway Server updates the gateway statistics every 4 to 6 minutes (random to distribute load). The RX and TX rates indicate the packets per hour. This is only indicative for the busyness and liveness of gateways. Although you can, I would discourage polling these values and converting them back to actual numbers (based on consecutive updatedAt timestamps), because you’ll be missing updates and with many gateways you’ll be hitting the API rate limiter.

We are working on a separate API for routing statistics, including per-gateway aggregates over time.

1 Like
17

Very useful API, thanks. My gateway moved (Antenna location changed thanks to GPS + Location correctly set in console). How long will it take to get the new value through the API ? (Is it minutes, hours, days? )

18

That should be minutes, up to 6 minutes roughly.

19

First of all: Great to have this API !

You mention that people should cache the results when they query the API.
→ Would it be possible to add an ETag header in the HTTP response?
This way a client cache can verify if certain data/content has changed without overloading the API by doing a simple HEAD request and not af full GET request.

20

ETag is a bit tricky, but Last-Modified can be used so you can pass If-Modified-Since instead. I’ll file an internal issue.

1 Like