TTN-Mapper and Lilygo-T-Beam

[Julian-Thiele]

Hello everyone, i have a few questions. It is generally about the TTNMapper. I recently switched my gateway to V3. In general, it all works too. The nodes that are connected send their data and I can see them in the console. Now I have a Lilygo T-Beam V1.1 GPS tracker. With this I wanted to explore the area a little, where there are gateways and where I can receive. Unfortunately, I don’t get it displayed on TTNMapper. Likewise, my V3 gateway is no longer displayed. Regarding the gateway: I entered the gateway as publish location under location and entered the coordinates. The Gateway is a Mikrotik Lora8Kit

Now to the T-Beam GPS tracker.
Data arrives cleanly and is also decoded. Only these do not arrive at TTNMapper either.

and here are the complete data

{
  "name": "as.up.data.forward",
  "time": "2021-04-24T08:31:43.138949783Z",
  "identifiers": [
    {
      "device_ids": {
        "device_id": "t-beam-julian-abp",
        "application_ids": {
          "application_id": "t-beam-julian-abp"
        },
        "dev_addr": "260B575D"
      }
    }
  ],
  "data": {
    "@type": "type.googleapis.com/ttn.lorawan.v3.ApplicationUp",
    "end_device_ids": {
      "device_id": "t-beam-julian-abp",
      "application_ids": {
        "application_id": "t-beam-julian-abp"
      },
      "dev_addr": "260B575D"
    },
    "correlation_ids": [
      "as:up:01F41G32GWHEA6EZD3DEW6FZM2",
      "gs:conn:01F40NVAW87MY5FJ607J0D3WW1",
      "gs:up:host:01F40NVAWCG6904E3RZY4TJVJ4",
      "gs:uplink:01F41G3293VKAKPKQMQNXY1TYS",
      "ns:uplink:01F41G329418FNSV0BERC64PA5",
      "rpc:/ttn.lorawan.v3.GsNs/HandleUplink:01F41G32948GGDB7D4JTAVSXHH",
      "rpc:/ttn.lorawan.v3.NsAs/HandleUplink:01F41G32GWMPJCTHCKTQBQP7PP"
    ],
    "received_at": "2021-04-24T08:31:43.136148257Z",
    "uplink_message": {
      "f_port": 10,
      "f_cnt": 606,
      "frm_payload": "yMGbhhbqAhQJCQ==",
      "decoded_payload": {
        "Altitude": 532,
        "Hdop": 0.9,
        "Latitude": 51.15676171521912,
        "Longitude": 8.56338194390429,
        "Sats": 9
      },
      "rx_metadata": [
        {
          "gateway_ids": {
            "gateway_id": "gateway-julian-lora",
            "eui": "3133303738006600"
          },
          "time": "2021-04-24T08:31:41.358996Z",
          "timestamp": 3564040367,
          "rssi": -39,
          "channel_rssi": -39,
          "snr": 11.25,
          "location": {
            "latitude": 51.15662791203662,
            "longitude": 8.561756038390223,
            "altitude": 520,
            "source": "SOURCE_REGISTRY"
          },
          "uplink_token": "CiEKHwoTZ2F0ZXdheS1qdWxpYW4tbG9yYRIIMTMwNzgAZgAQr+G7ow0aDAjuro+EBhCxoaalAyCY1/CK3dUG",
          "channel_index": 1
        }
      ],
      "settings": {
        "data_rate": {
          "lora": {
            "bandwidth": 125000,
            "spreading_factor": 10
          }
        },
        "data_rate_index": 2,
        "coding_rate": "4/5",
        "frequency": "868300000",
        "timestamp": 3564040367,
        "time": "2021-04-24T08:31:41.358996Z"
      },
      "received_at": "2021-04-24T08:31:42.884626680Z",
      "consumed_airtime": "0.370688s"
    }
  },
  "correlation_ids": [
    "as:up:01F41G32GWHEA6EZD3DEW6FZM2",
    "gs:conn:01F40NVAW87MY5FJ607J0D3WW1",
    "gs:up:host:01F40NVAWCG6904E3RZY4TJVJ4",
    "gs:uplink:01F41G3293VKAKPKQMQNXY1TYS",
    "ns:uplink:01F41G329418FNSV0BERC64PA5",
    "rpc:/ttn.lorawan.v3.GsNs/HandleUplink:01F41G32948GGDB7D4JTAVSXHH",
    "rpc:/ttn.lorawan.v3.NsAs/HandleUplink:01F41G32GWMPJCTHCKTQBQP7PP"
  ],
  "origin": "ip-10-100-13-61.eu-west-1.compute.internal",
  "context": {
    "tenant-id": "CgN0dG4="
  },
  "visibility": {
    "rights": [
      "RIGHT_APPLICATION_TRAFFIC_READ"
    ]
  },
  "unique_id": "01F41G32H26WF1J6CYNSD8PXCW"
}

Now also the Webhook:

Who can help me?

[Jeff-UK]

@Julian-Thiele

I recently switched my gateway to V3

See

TTN V2 to V3 migration

Packet Broker currently routes traffic from gateways connected to The Things Network V2 to V3. That is, gateway coverage of V2 is available in V3. But not the other way around : gateways already connected directly to V3 will not be available for applications in V2. As a friendly request, both as a user of TTN and as the maintainer of TTN Mapper, please keep your gateways on TTNv2, until as late as possible. People in your city that have not yet migrated their application from v2 to v3 will…

[Julian-Thiele]

this is right, but i have some Nodes with ABP. When the Gateway run in V2 and the Node is bei ABP in V3 there are no Data. Its running, when the gateway and Node is on v3

[Julian-Thiele]

oh no its running… But i have no Data on TTN-Mapper

[wolfp (Wolfgang)]

imho there is no TTNmapper integration for V3.

[Julian-Thiele]

Hey Wolfgang. For the Gateway or the GPS-Tracker? A Friend use a Dragino LGT-92-Li on V3 and he can Track to the Mapper

[Jeff-UK]

@Julian-Thiele

A Friend use a Dragino LGT-LGT on V3 and he can Track to the Mapper

As @wolfp calls out there is no official TTN V3 integration as yet. Last week’s 3.12.0/3.12.1 update to TTS CE potentially allows for a specific kludge through webhooks that allows a limited set of functionality (via android on smart phone or gps enabled node?) as I understand it (you may need to go read github details of TTS update and TTNMapper to get understanding), so perhaps your friend has gone that route and can help you? A formal Template to allow full integration is still some way off (waiting for it myself!), perhaps JP can update on status @jpmeijers ?

[jpmeijers]

The Webhook Template is the “official” integration between The Things Stack v3 and TTN Mapper. I know it feels like a kludge, but this is how things work in V3.

You say it only has a subset of the functionality. What is missing?

Mapping for V3 works exactly like for V2:

• either using any device with a mobile phone (iOS or Android).
• or using a GPS tracker with the integration on the console

TTN Mapper Documentation

For now all V3 mapped data should appear when one filters for a specific device under advanced maps. The aggregated layers do not yet use this data, except the heatmap. Processing of the data is still under development and not as trivial as with TTNv2.

[Julian-Thiele]

Hey @jpmeijers, thanks for your answer. I have two problems.
1: I can’t see any Data from my T-Beam at TTNMapper.
At the V3 console there are the Datapoints (see my first post). I have also integrated the Webhook (see also my first Post).
When i now under Advanced Maps my ID “trackerjulian”, no data is displayed. I am now assuming that something is wrong with the translation of the payload. Could you check whether the data as described above is correct? Or do I have to do something differently?

2: When I run my gateway under V2, it is shown on the map. As soon as I run it on V3, it is no longer displayed on the map. Under Loaction I set the gateway to Publish. I also entered coordinates (once exactly the same as under V2, once at least 150m further away). Does anything else have to be done here?

best regards
Julian

[Jeff-UK]

Hi JP, sorry if there is a misundestanding here - appreciate Webhook Template may now be the/an official route under V3, I was working off your most recent posts in

TTN Mapper and how to indicate V2 vs V3 vs Private

Many questions, so let me answer the core ones. A gateway is uniquely identified by the network id and the gateway id. So there is a TTNv2 gateway with ID eui-0011223344556677, and there is a separate gateway on eu1.thethings.network (V3 eu) with ID eui-0011223344556677. These two are currently handled separately. At some point we can try and find a way to either copy old coverage data over, or merge the two. We can do this because we know TTNv2 and eu1.thethings.network are essentially the sam…

Were you said
“Mapping V3 is already possible using the Android app. It is also possible using a gps tracker and setting up a webhook integration. It’s a little tricky to do, and therefore we need a template. Issue is that the template doesn’t allow optional fields, making the “experiment” option difficult to implement. My plan is to release a template that does not have the experiment option, but one can manually add it by editing the webhook.”

and later

" You will see on https://status.thethings.network/ that the update is scheduled to be deployed on 12 April. Only after that we can see how this was implemented, and add support for the new identifiers on TTN Mapper’s side".

I may have missed but haven’t seen any specific updates to say TTN Mapper integration now done and complete so assume(d) still a work in progress and not yet fully tested/complete.

Are you saying that original request and advise in

TTN V2 to V3 migration

Packet Broker currently routes traffic from gateways connected to The Things Network V2 to V3. That is, gateway coverage of V2 is available in V3. But not the other way around : gateways already connected directly to V3 will not be available for applications in V2. As a friendly request, both as a user of TTN and as the maintainer of TTN Mapper, please keep your gateways on TTNv2, until as late as possible. People in your city that have not yet migrated their application from v2 to v3 will…

no longer holds from TTNMapper POV (still holds for other use cases and due to impact on other local community V2 users of course) and that you are happy for all to migrate to TTN Mapper using V3?

@jpmeijers

What is missing?

Speaking as a non-softie and someone who only rolls sleeves up and codes (more usually copy, paste/edits) when forced too (!) you could say what is missing is an ‘easy’ integration such as where in V2 we could ‘push the button’ for V2 integration of Cayenne/MyDevices, Data Storage, or TTN Mapper etc, where as (with exception of Data Storage) under V3 we no longer have that option… would be good to have added and that is what I may have misunderstood as missing and might be enabled by an official Template…I know there are many others like me who would like an easy path “click to enable this” option Also as highlighted by Julian above, there still seems to be some missing functionality and behaviours, which I thought indicated not yet a production solution, again apologies if I have got this wrong or misunderstood.

@jpmeijers

Processing of the data is still under development and not as trivial as with TTNv2.

Please keep us all updated on TTN Mapper development status (do we need a dedicated thread for that under the V3 Migration category so details dont get lost or peanut buttered across other discussions?)

Update: May also be worth adding a link to the docs as above to the TTN Mapper home page menus (I dont see it come up there or under any subheadings)

[Jeff-UK]

@Jeff-UK

I know there are many others like me who would like an easy path “click to enable this” option

@jpmeijers JP Ignore me as just revisited the Webhooks integration option for a V3 tracker I had set up a month or two back, prompted by now seeing your documentation, and where the Webhooks listing is blank I now see if I click ‘add integration’ that there is indeed now an option to click and add TTN Mapper integration - I can go play now ! I dont recall seeing that when I looked previously… You may wish to publish more widely - perhaps using a dedicated TTN Mapper/V3 thread per my suggestion…

[Julian-Thiele]

Hey at all,
does anyone else have any idea what to set to make it appear? I followed the instructions. But unfortunately we haven’t had any success yet.

[RockiesIOT]

I have a new gateway registered in v3 and a device with the TTN mapper intergration. Neither device shows up in TTN Mapper even when searching for the specific device ID. The gateway was registered in v2 and used to show data. I deleted the gateway and re-added it in v3.

If I search the gateway eui in Advanced Maps I see the old data.

I’m guessing that the move to v3 is a great deal of work and that it will eventually work. Meanwhile I’m patient and appreciate the work everyone is doing.

[engelking (hek)]

ttnmapper.org needs to be modified in many aspects to work with TTSv3.
As far as i know only in the heatmap new v3 datapoints will be shown.
Everything else is v2 only.