Inaccessible gateways

Hi,
I have trouble with migration my gateways and devices to V3. I have more than 50 GWs all around the country, with no remote control (no VPN, no public IP address, physically hard to access them). There are connected to router.eu.thethings.network as packet broker. If I register each of these gateways via console to V3, there are disconnected.
Only one thing, which I need from TTN community, is make CNAME in DNS from router.eu.* to eu1.cloud.*
Is it is possible? Because otherwise all these gateways will be erased from network and we can’t collect data!

That’s half the job - the other half is to change the address on the gateway to point to the v3 servers.

The community has NO access to the TTI infrastructure what-so-ever. But fortunately you will see that the v2 gateways are actually handled by the Packet Broker to route them through so a v2 gateway will continue for a while yet.

But you will still have to get physical access to the gateways as some point over the coming months - we’ve had ~9 months to do this, no one knows how long TTI can keep the old v2 gateway to v3 routing in place, so best start planning.

My question is, why TTN can’t reroute old address to the new one? Why TTN can’t made DNS entry with CNAME eu.router.* to eu1.cloud.*
It’s so easy and without any additional costs.

Perhaps you could ask them on Slack - as TTI don’t frequent these parts on the timescale you are hoping they will.

Gateways connected to V2 endpoints will keep forwarding traffic to V3 networks through Packet Broker for some time after V2 shutdown, so they won’t go offline on December 1st.

There are several reasons why we won’t just create CNAMEs, here are some of them:

  • The current solution with “legacy infrastructure” that forwards traffic to Packet Broker works fine.
  • It’s actually a good thing for people to upgrade the firmware of their gateways while they’re re-configuring them. And if you don’t have secure remote access, you should set that up anyway.
  • Not everyone wants to migrate their gateway to eu1.cloud.thethings.network. If we keep these gateways on the “legacy infrastructure” it’s easier to migrate to the network of their choice instead of being forced onto eu1.cloud.thethings.network.
  • V3 requires all gateways to be registered, whereas V2 (and the “legacy infrastructure” that forwards to Packet Broker) allows unregistered gateways, and we have quite a few of those. If we would just create a CNAME to V3, all those unregistered gateways would immediately disappear from the network, and we want to avoid that. We don’t have a final solution for these gateways yet, maybe we will eventually make the old DNS records point directly to Packet Broker.
1 Like

Hi Hylke,
but problem with this solution is with migration OTAA with active session.
If gateways are still in V2 and I want to migrate my V2 application (devices) to V3 (with keeping active sessions), it’s not worked.
Is there some advise, how could I deal with it?

Migrating ABP devices (or OTAA sessions) is only possible if you also migrate your gateways. If you can’t migrate your gateways, you need to migrate your OTAA devices without session and make them do a new join on V3.

And this is the problem. Inaccessible gateways and inaccessible OTAA devices.
Ok - if i migrated application with keeped active session via CLI, is there a chance, that OTAA devices will in couple of days try rejoin on V3 (via eu.router.thethings.network packet broker)? Or not?

If you completed the migration process with ttn-lw-migrate, you should now be in the following situation:

  • Your end device was registered as an OTAA device in V3
    • The next time your device joins, the V3 network will accept that join.
    • (for completeness: if you also migrated the session and your device is covered by gateways on V3, the device should work immediately)
  • Your end device was converted to an ABP device in V2
    • The next time your device joins, the V2 network will not accept that join.
  • Your end device was disabled in V2 (if you migrated with session)
    • The next time your device sends a (confirmed) uplink, the V2 network will not respond.

When a device joins, depends on the implementation of the end device, so check the manual of your end device (if you bought it off the shelf) or the firmware (if you wrote it yourself). Ideally you can send a downlink command to reset the device. Some devices reset if they don’t hear back from the network for some time. And if you’re really unlucky, the device only resets if you physically remove the battery, and put it back in after 30 seconds.

No and Yes.

If the active session transferred OK & the device is disabled in v2 you would not see any interruption in uplinks. As you are hoping they will rejoin, this clearly hasn’t worked.

If a device has gone quiet either one of two things will happen:

  1. The Link Check or similar functionality will kick in and a rejoin will occur.
    Or
  2. You reset / power cycle the devices.

The first is all about the firmware - you should ask the manufacturer(s) about that.

The second is a physical thing, however unpalatable it may be to ask customers to do that for you and then visit the rest to mop up the rest.

Let’s presume, that devices HAVE functionality to re-join after lost connection (it’s water and energy meters, can’t be accessed by me, can’t be restart by customers, can’t be interrupt power by battery)

My question is: IF I made ttn-lw-migrate with parameter --ttnv2.with-session=true, AND my GW was NOT successfully migrated to TTNv3 (still send data to eu.router) is there chance, that device will re-join to V3 application as well as i migrated it with parameter --ttnv2.with-session=false?

You will need to ask the manufacturer(s) if the device(s) support Link Check or has something similar in firmware.

Without a reset / power cycle, that is your only possibility that I’m aware of.

Hopefully this will nudge you in to asking the manufacturer(s) before hoping against hope that someone will come up with a solution. We’ve been doing this migration Q&A for many months now, if we had some magic in our hats, we would share it with you.

Hi @nufinuf, I think I have some good news for you: Update on DevAddr Block Assignments on The Things Stack Community Edition clusters.

With this change, traffic from your end devices that still have V2 DevAddrs, received by your gateways that are still connected to V2, should now get forwarded to V3.

Thanks God.

One more thing - if I migrate some of applications from V2 WITHOUT keeping session active and now I need to KEEP the session and adding keys into v3, is there some possibility, how to recover these keys from V2?

In my account, it is applications (in V2): go-celakovice, go-havanska, go-hradec, go-humpolec, go-lukavec, go-praha, lm-khory, lm-nadpetruskou, maddeo, polska

There I type ttn-lw-migrate application --source ttnv2 "go-celakovice" --ttnv2.with-session=false, so keys from V2 console were rewritten.

If you used ttn-lw-migrate to migrate devices without session, the end device registration in V2 was converted to ABP (unsetting the AppKey), leaving the session alone. So those session parameters should still be in V2.

Yes. But all devices are OTAA. And I need all these keys, including session_key_id. Without them I can’t migrate to V3 previous session with OTAA settings :frowning: In V2 is not session_key_id now :frowning: