How to Migrate ABP Devices from V2 to V3

TTN V2 to V3 Upgrade Migrating Devices to V3
1

I’m writing this post as a quick draft that will be extended and eventually turned into a documentation page.


:warning: Migrating existing ABP devices requires one of the following:

  1. Your gateway is connected to V3, or
  2. You update the device address (DevAddr) in your end device(s) with a new device address generated in the V3 console. This option may require flashing new firmware to the device.

Read below instructions carefully. If you make a mistake it might not be possible to correct it later. If so, a new device will have to be added instead and the current device ID will become unusable because it cannot be re-used.

Before you Begin

  • Using ABP devices is discouraged. We recommend to use OTAA unless you really have a good reason to use ABP.
  • ABP devices are (by definition) hard-coded (personalized) to a specific Network Server, so it’s technically not possible to migrate them to a different Network Server.
  • If you do migrate ABP devices, they will keep their V2 DevAddr. Packet Broker will not route traffic for these devices to V3. This means that you will have to connect your gateway to V3.
    (Note: a gateway connected to V3 will not forward traffic from V2 devices to V2).
  • ABP devices on V2 use an RX delay of 1 second. V3 uses an RX delay of 5 seconds. V3 will try to send this configuration to the end device in a donwlink MAC command, but still, downlink may be unreliable for these devices.
  • You may have to reset your end device if it has a high frame counter.
  • If you have easy access to your device, it’s probably better to just re-program it. This would be a good opportunity to update the firmware to switch to OTAA and add some best practices.

1. Migrating a Few Devices

If you’re migrating a few devices, you can just do that in the Console:

V3 Registration

  • If you don’t have an application in V3, add a new one.
    • The Application ID does not have to match your V2 application ID.
  • Add a new End Device in V3.
    • Follow the manual steps:
    • Select Activation by personalization (ABP)
    • The LoRaWAN version is probably v1.0.2.
    • In the Basic settings step, the End device ID does not have to match your V2 device ID.
    • The DevEUI is optional.
    • In the Network layer settings step, Select a Frequency Plan. For EU868 devices coming from v2, you select Europe 863-870 MHz (SF9 for RX2 - recommended).
    • The Regional Parameters version is probably v1.0.2 rev B
    • Your device does not support class B or class C
    • The DevAddr and NwkSKey must be exactly the same as for your v2 device registration.
    • The Advanced settings must be set on registration.
      Changing these settings later may not work.
      • The Frame counter width is probably 32 bit
      • The RX1 Delay is 1 second
      • The RX1 Data Rate Offset is 0
      • The device probably resets frame counters
      • The RX2 Data Rate Index for EU868 devices coming from v2 is 3
      • The RX2 Frequency for EU868 devices coming from v2 is 869525000
      • The Factory Preset Frequencies for EU868 devices coming from v2 are:
        • for all devices:
          868100000, 868300000, 868500000
        • for devices that use 8 channels
          867100000, 867300000, 867500000, 867700000, 867900000
          (in addition to the 3 channels above )
    • In the Application layer settings step, the AppSKey must be exactly the same as for your v2 device registration.

Deleting the Device from V2

With the registration in v3 set up, you need to prevent the V2 Network Server from handling traffic for your device. You can do that by deleting the device from V2.

Connecting Your Gateway to V3

Because your device is still using a V2 DevAddr, Packet Broker won’t route traffic for it to the V3 Network Server if your gateway is still connected to V2, so you have to connect your gateway to V3.

2. Migrating a Lot of Devices

If you need to migrate a lot of devices, you can use the migration tool.

[to be added later]

3 Likes
2

Well, by creating a new device, it seems possible to have an ABP v3 working through GWs in v2. If my understanding is correct, we should ask people not to migrate their gateways for the moment…

3

This guide is specifically for migrating existing devices, not about new devices. When creating new ABP devices on V3, you can request a DevAddr from the Network Server, and Packet Broker routing (at least of uplinks) will work fine. If the firmware of your ABP device is configured with an RX1 Delay of 1 second (which it is, unless you changed it), downlink through Packet Broker may not work (it may not arrive in time for that 1 second window).

People are free to migrate their gateways, or to keep them on v2. We do ask people to coordinate the migration of gateways with their local community.

And I can’t repeat this often enough: We’re asking people to not use ABP devices unless they really have a good reason to use ABP.

1 Like
4

Thanks for confirming these settings. From previous posts I learned that MAC state is not going to set the device with downlink messages. Unfortunately, I still see downlink messaged sent by V3. Is this a malfunction or am I missing something?

6

Ok, by reprogramming the params in the device (DevAddr etc.) I was able to connect a device via ABP and my V2 gateway to V3. I would reccommend to publish your helpful tips more prominent in the TTN documentation.

Nevertheless I think the process is unfortunate:

  • I essentially need to update all devices one by one to be sure not to have trouble when I update the gateway. And then change the payload decoding and maybe (haven’t checked yet) the integration.
  • My devices are low-cost sensors with little demand for security but they should be cheap. OTAA costs too much overhead in the libraries for that. So please keep ABP. Reccommendations in the forum to just switch to new hardware is not an option, I would rather find a software solution and leave TTN.
  • I think the community of amateurs will have quite a lot of work with this for essentially no relevant benefit. For this, the transition period of less than a year is way too short.
  • So it feels a bit the amateurs are not wanted anymore and you focus more on industrial solutions. Fine, but then it should have been communicated differently.
3 Likes
7

Once you have tested a few, you should be able to use the migration tools which we are all working on to improve - you could too. Payload decoding does not have to change, V2 works in V3. Hard to comment on the integration without you saying what you are using.

Sure, some have said up date your hardware. Equally some of us are saying that if you give it a bit of time, collectively, we can find solutions.

Your hardware is for LoRaWAN, the firmware is all configured. Yes, you have to move over the V3, tools being improved to help with this, but changing network, assuming it has enough coverage and provides the extensive backend server free of charge, may well require you to physically access the device and change the firmware.

The V2 stack / servers are a drain on TTI resources. This is a free service, why would they carry on running V2 when they have a much better solution coming on stream?

If we were told the deadline was the end of 2022, we’d just be having this discussion in Sept 2022. If you have that many devices you need more than a year to migrate, perhaps you could fund the migration.

TTN V2 provided a fantastic proof-of-capability as well as being a huge benefit to a whole raft of smaller non-commercial users (and even some commercial ones). It’s an old stack, it needs to move on, like any other IT system, upgrades are part of the landscape. We’ve been given our own set of servers with two more on their way and more to come. Hardly the approach of a company that doesn’t want us anymore. They could have provided TTN V3 on a small subscription basis to cover the hard costs. But they haven’t. Yes, TTI get the benefit of proof-of-capability from us and we get some kick-ass state of the art LoRaWAN facilities with some technical & development staff that live & breath LoRaWAN.

Last point, from the keynote (& my memory) about 20% of traffic is TTI with 600 packets per second. So TTN is getting 480 packets processed a second for free, and, sadly I know from a moderators meeting, rather too many downlinks. We get so much more than we are asked to put in other than buying and running a gateway.

1 Like
8

We are monitoring as of today the migration progress of our community members at TTN Apeldoorn: https://twitter.com/RFSeeTweets/status/1359431645271588870

2 Likes
9

Thats good to know Remko. Like the recent stats #'s - enlightening.
image

Perhaps you could do a short write-up and post a ‘how to’ so other communities can replicate efforts to assist & manage migrations?

Mind you a pity about this not being fully open (not everyone wants to be in the clutches of the claws of the little bird! :wink: ) :+1: :-
:
image

1 Like
10

It is as open as it gets. This information is also posted at the community Slack of TTN-Apeldoorn. You will get assimilated anyway :scream:

11

:+1: Thats also good to know

…Resistance is NEVER futile! - Live long and use LoRaWAN :wink:

1 Like
13

To be clear: The Things Stack (v3) still supports ABP devices, and we have no plans to remove ABP from The Things Stack.

We have been discouraging using ABP in production because it comes with a lot of pitfalls. Most notably, it’s difficult to switch to another network, which is essentially what we’re doing when migrating from v2 to v3.

2 Likes
14

@htdvisser
Can you add the following information to the guide in the OP?:

It is good to have a fallback option in case the migration does not go as expected.

What works best for me is to change the device’s NWSkey in V2 to disable the device in V2.
(changing only the last byte is already sufficient).

If the migration works as expected the device can then be deleted from V2, if not then the NWSkey can be restored to make the device work on V2 again.

Such information is currently lacking from the guide (OP).

1 Like
15

@htdvisser @johan

  1. “Three frequencies are defined for all devices.”
    • This is not clear at all from the console (see screenshot below)
      The console shows no information about this whatsoever.
    • What does “List of factory-preset frequencies” mean here?
      Does it mean ‘the list of channel frequencies programmed into the device’?
      (For many community members who build DIY devices, ‘factory preset’ may be confusing.)

Image 2021-02-11 12·17·51

  1. The console for Advanced Network Settings does not show any default values but it should.
    Leaves the user wondering without providing any information.
    It should be clear what the default values are.

  2. “for all devices:
    868100000, 868300000, 868500000”

    • What is the reason that V3 defines only 3 frequencies for devices?
      Why not 8 like is the case for V2?

  3. “for devices that use 8 channels”

    • Don’t all normal devices use 8 channels?
    • How should a (migration) user know whether (s)he needs to provide frequencies for 8 channels or not?
    • Must the user add only the 5 additional frequencies, or all 8?
      Does explicitly entering a frequency automatically clear the 3 default defined or not?
      This is not made clear in above description and the console does not provide any information at all.

  4. Why does the user have to manually enter the list of frequencies (for devices using 8 channels)?
    Having to manually enter them is not only impractical but also error prone. In V2 the user did not need to manually add frequencies.

16

I have run the numbers for 24 hours for the first time.

I have analysed all data received by 12 gateways that are operational in our area:
Over 24 hours I have observed 3581 unique device addresses.

Note that through, for example, regular OTAA attempts the numbers are not exact bu give a good indication.

Devices heard (network, count, percentage of total):

  • TTN: 912 (25%)
  • KPN: 1915 (53%)
  • Helium: 8 (0,2%)
  • Other networks: 746 (20%)

Of the TTN devices:

  • 881 use V2 (98%) (43 of these (5%) use ABP)
  • 17 use V3 (2%)

There is a lot of work to do!

3 Likes
17

This according to LoRaWAN EU868 specification; all devices start with the three join frequencies and they get the other channels via the join-accept.

They use them, but they get them from the network.

This should happen automatically with migration tooling.

All 8.

Note that ABP devices in EU868 can be configured in two ways:

  1. Only with the 3 default mandatory channels. In this case, V3 configures the other 5 channels via MAC commands
  2. With the 5 additional channels. If this is the case, V3 will not issue MAC commands for the additional channels are they are assumed to be there via the factory preset frequencies

That’s why this is a required option.

There are in total 8190 DevAddr that TTN V2 EU issues (it was 4096 before), so there’s a bunch of duplicates here. I recall seeing sometimes 25 duplicates on some DevAddrs.

How do you know?

18

Not for users manually adding their (former) V2 devices to V3.

So how should these users know?
The console does not give any hints or instructions for this.It also does not make clear that all 8 frequenties must be entered.


What do you mean with ‘this’ here?


Ok, clear: setting the 5 additional frequencies in the console (which requires entering all 8 frequencies) prevents sending MAC commands to the device for setting the additional 5 channels.

Looking at the MCCI LMIC (and predecessor) library’s ttn-abp.ino example, for ABP (EU) always 8 channel frequencies have been set explictly.

  • Is the configuring of the 5 other channels via MAC commands implemented in V3 but was not in V2?

  • If setting the additional 5 channels can be done via MAC commands why do there exist 2 situations: 3 factory preset and 8 factory preset channels?
    Is this because additional channels via MAC commands was added in a later version of the LoRaWAN specs or for other reasons?

ADR shall be used for stationary end devices only, not for mobile devices in transit.
What about setting the additional 5 channels via MAC commands, should this be prevented for mobile devices also and always set the 8 channels in the device and in the console for mobile devices?

19

I analyzed the device address range using the information I got from this forum, @htdvisser and https://github.com/TheThingsIndustries/lorawan-stack-docs/issues/215#issuecomment-770800712 as well as https://www.thethingsnetwork.org/docs/lorawan/prefix-assignments.html

So true. I never intended to be exact. An estimated error on the numbers could be about 1% so the numbers are significant.

There is no need to argue the numbers. The message is quite clear: Let’s start migrating!

20

@neoaggelos @roman please comment on this.

We’re tracking this issue:

https://github.com/TheThingsNetwork/lorawan-stack/issues/3785

That there are two ways, and that any LoRaWAN network server needs to explicitly know if the device uses factory preset frequencies that are different from the default set of join channels.

Note that LoRaWAN networks can also choose not to use the three join channels once devices joined the network, to dedicate those channels to device activations (and other networks). This is why the factory preset frequencies are all the channels and are independent of LoRaWAN join channels.

Yes

On another note, @snejra made lots of progress on the migration documentation on thethingsnetwork.org/docs, see https://github.com/TheThingsNetwork/docs/pull/425

We very much welcome clarifications there. I highly appreciate articles and forum posts, but ultimately we need to get this documented.

2 Likes
21

@htdevisser I see no mentioning of copying AppSKey in your instructions.
I assume this needs to be copied from V2 settings as well because that is the AppSKey that is programmed into the device. Correct?