Host `cluster` failed to handle message. No devices matches uplink

[tomldev]

Hello,
I just added my gateway to the new v3 stack and now I get a lot of the messages:

Host cluster failed to handle message. No devices matches uplink

    {
  "name": "gs.up.drop",
  "time": "2021-02-24T11:50:02.378887556Z",
  "identifiers": [
    {
      "gateway_ids": {
        "gateway_id": "3436323825005f00",
        "eui": "3436323825005F00"
      }
    }
  ],
  "data": {
    "@type": "type.googleapis.com/ttn.lorawan.v3.ErrorDetails",
    "namespace": "pkg/gatewayserver",
    "name": "host_handle",
    "message_format": "host `{host}` failed to handle message",
    "attributes": {
      "host": "cluster"
    },
    "cause": {
      "namespace": "pkg/networkserver",
      "name": "device_not_found",
      "message_format": "device not found",
      "correlation_id": "35d474cb02b349f39703b1e02542d3b8",
      "cause": {
        "namespace": "pkg/networkserver/redis",
        "name": "no_uplink_match",
        "message_format": "no device matches uplink",
        "code": 5
      },
      "code": 5
    },
    "code": 5
  },
  "correlation_ids": [
    "gs:conn:01EZ9TNJ4PEHJZRYBXNQHGQXCS",
    "gs:up:host:01EZ9TNJ4YN782NBESKSFAKJ2S",
    "gs:uplink:01EZ9XZSW663RXPGRE44RCZ5F7"
  ],
  "origin": "ip-10-100-5-46.eu-west-1.compute.internal",
  "context": {
    "tenant-id": "CgN0dG4="
  },
  "visibility": {
    "rights": [
      "RIGHT_GATEWAY_TRAFFIC_READ"
    ]
  },
  "unique_id": "01EZ9XZSWA7NMRM8MA6ZK6706R"
}

With the v2 stack it seems to work without issues.
Why is that? What does the message mean?

[descartes (Nick McCloud)]

Probably all those v2 devices in your area that are transmitting but aren’t setup on v3 so your gateway passes on the uplink but there’s no entry to match in v3.

Which is why we are suggesting people hold off migrating their gateways to v3 until more v2 devices have been moved.

[vaasseniot]

And not every device is configured to be a TTN V2 or V3 device also other carriers. If you use packetforwarder then every device that sends within the bandwith wil be forwarded to TTN if configured.

[4commerce-technologi (4commerce technologies AG)]

Hi, I also moved our gateway and all devices to v3.
With those everything is working fine!

But also a number of device ID’s will raise the error “Host cluster failed to handle message”.

May I filter this Device-IDs within my gateway?
I am using a Dragino LG308 with latest firmware applied.

Thanks for help.

[4commerce-technologi (4commerce technologies AG)]

Just to be more precise:

I wish to block / filter unknown [DevAddr] which cause errors only.

[kersing (Jac Kersing)]

@4commerce-technologi

But also a number of device ID’s will raise the error “Host cluster failed to handle message”.

My I filter this Device-IDs within my gateway?

You should not filter. Those devices might be from another TTN user that still needs to migrate their application (and might now be missing data because you migrated your gateways early, the TTN console explicitly instructs to migrate applications/devices, not gateways) and once they migrated their data should not be filtered.

@4commerce-technologi

I wish to block / filter unknown [DevAddr] which cause errors only.

Those messages are not errors. It is merely an informational statement that the community network has no destination for that device address (combined with active Network Session Keys). That message can be useful when debugging connection issues with devices.
In any case, your gateway does not know which devices are known within TTN, it shouldn’t. The gateway should just pass all traffic to the backend and the backend can decide what needs processing and what can be ignored.

[4commerce-technologi (4commerce technologies AG)]

Well @kersing

thanks for your feedback - in that case - we will stay with that and “ignore” the messages when debugging our gateway and/or sensors traffic.

[descartes (Nick McCloud)]

You could put your gateways back on v2 and be a good TTN citizen …

[4commerce-technologi (4commerce technologies AG)]

Hey Nick @descartes

not sure but does this mean - sensors on v3 will be able to communicate over gateways on v2 ?

Btw., I thought “I am a good TTN citizen” going to provide and support v3

[descartes (Nick McCloud)]

@4commerce-technologi

not sure but does this mean - sensors on v3 will be able to communicate over gateways on v2 ?

Yes

@4commerce-technologi

Btw., I thought “I am a good TTN citizen” going to provide and support v3

No - way too early to move as discussed & linked all over TTN site & forum - v2 devices can only be heard by v2 gateways. So anyone who has/had a v2 device being picked up by your gateways (the TTN deal being they are community accessible) has either one less gateway or, depending on location/coverage, has NO gateways to hear them.

[4commerce-technologi (4commerce technologies AG)]

@kersing @descartes

Sorry for my very last clarifying question:

This does mean - v3 Gateways are at least not needed currently while not all devices are moved to TTN v3 devices and applications?

Hmmm … but … wouldn’t be moving the gateways to v3 will FORCE the device owners to move forward to v3 also?

I am asking this because - I always ask myself why it took such a long time to move e.g. from IPv4 to IPv6 even that is an easy path. As long as old things are supported - I know a lot of admins / owners won’t spent their time in upgrading. That’s a pity from my point of view!

At least - the TTN v3 gateway console is an improvement for us as well.

[cslorabox]

@4commerce-technologi

Hmmm … but … wouldn’t be moving the gateways to v3 will FORCE the device owners to move forward to v3 also?

If you mean “force” in the sense that things that used to work may stop doing so, then yes.

I am asking this because - I always ask myself why it took such a long time to move e.g. from IPv4 to IPv6 even that is an easy path.

It wasn’t actually a trivial change, with many things being unable to readily support it. And it was effectively an optional change, because:

As long as old things are supported

If you move a gateway to V3, then V2 devices are not supported.

[4commerce-technologi (4commerce technologies AG)]

@cslorabox

If you mean “force” in the sense that things that used to work may stop doing so, then yes.

Yes - and sorry - but that’s what most people will push and start to move forward …

@cslorabox

If you move a gateway to V3, then V2 devices are not supported.

Got that

@4commerce-technologi

This does mean - v3 Gateways are at least not needed currently

BUT then - for what and why has TTN enabled the configuration of TTN v3 Gateways. If they won’t enabling the migration path - they just would had left that option disabled?

[descartes (Nick McCloud)]

@4commerce-technologi

This does mean - v3 Gateways are at least not needed currently while not all devices are moved to TTN v3 devices and applications?

Until the bulk of people have wrapped their heads around the logistics and then actually done the move, we are recommending people leave their gateways until v2 goes read only. They will carry on working but at that point it will encourage people to transition any remaining v2 devices.

@4commerce-technologi

wouldn’t be moving the gateways to v3 will FORCE the device owners to move forward to v3 also?

Yes, if they have the time & capability - they may not have finished planning & testing. Forcing them isn’t helpful at this point in time.

@4commerce-technologi

I always ask myself why it took such a long time to move e.g. from IPv4 to IPv6 even that is an easy path

So incredibly easy that all my corporate (national retailers with hundreds of stores, so really big, well funded, well resourced) clients are running their office networks on IPv6 - NOT.

@4commerce-technologi

At least - the TTN v3 gateway console is an improvement for us as well.

That’s subject to debate - this topic started because to some extent, it totally isn’t. Others have noted & requested filters to keep the detail manageable.

[cslorabox]

@4commerce-technologi

BUT then - for what and why has TTN enabled the configuration of TTN v3 Gateways. If they won’t enabling the migration path - they just would had left that option disabled?

For one thing, because features need real world testing.

Some may spin up additional gateways in the new system without moving their existing ones.

But it’s ultimately up to you if you want to move your gateway to V3 now; people have expressed their personal opinions against it, and given the reasons they reached that opinion, but in the end it is your choice.

Just like no one can tell you that you can’t unplug your gateway and put it in the closet.

[4commerce-technologi (4commerce technologies AG)]

@cslorabox

Just like no one can tell you that you can’t unplug your gateway and put it in the closet.

Another good option to test someday - at least the Dragino gateway firmware here in our local office is aweful

@descartes

Yes, if they have the time & capability - they may not have finished planning & testing. Forcing them isn’t helpful at this point in time.

@cslorabox

people have expressed their personal opinions against it, and given the reasons they reached that opinion, but in the end it is your choice.

Thanks to all the personal opinions - I like to be a fair player and from that point of view we will discuss internally to spin up an additional v2 gateway within the next days.

Cheers
Tom

[kersing (Jac Kersing)]

@4commerce-technologi

Hmmm … but … wouldn’t be moving the gateways to v3 will FORCE the device owners to move forward to v3 also?

TTN forces people to move later this year by making V2 read only and later by shutting it down. In the mean time you don’t know how much work is involved for others to move. For instance V3 does not have MyDevices Cayenne support. Anyone using that needs to find a new solution and migrate. That isn’t a trivial exercise if you don’t want to start paying many $$ a month.
That’s why in our community we agreed to move gateways as late as possible. New gateways are added on V3 for additional coverage (not that you’re able to show that because both TTNmapper and the main TTN websites map are not V3 ready yet).
That same advice is all over the forum as well… Not because we like it that way, but because migration is not trivial for everyone. (Some nodes need to be reset manually and in this COVID-19 world getting to them isn’t trivial either)

[cslorabox]

@4commerce-technologi

at least the Dragino gateway firmware here in our local office is aweful

Off topic, but you you can build your own from source thus fixing whatever you dislike.

(At least assuming you’re talking about one of the boxes that is an actual concentrator based gateway - the non-gateways things with one or two node radios may share the OS build but hardware problems like that can’t be fixed in software)

[4commerce-technologi (4commerce technologies AG)]

Hi Jac, hi Nick, @kersing @descartes

when we setup an additional Gateway for TTN v2 network and locate this right beneath the other v3 gateway, will that cause conflicts in radio communications? I have no experiences currently with two gateways directly in the same area.

Thanks for some hint
Tom

[cslorabox]

@4commerce-technologi

when we setup an additional Gateway for TTN v2 network and locate this right beneath the other v3 gateway, will that cause conflicts in radio communications?

Neither gateway will practically be able to receive while the other is transmitting, but that’s basically a given.

The more interesting question would be if the transmissions of one gateway could be strong enough to actually damage the other. That’s a bit uncertain - I know people who put gateways closer together than I’d like, they haven’t seen a problem, but…