Cannot export end devices using the migration tool

,
1

Hi there,
I have 2 environments set up, both running TTS 3.17.1

I’m trying to export devices from one environment, to then import into another environment by using the migration tool 0.9.0.

I keep getting this error:
Error: error:go.thethings.network/lorawan-stack-migrate/pkg/export:export (export device test-lux)
error:go.thethings.network/lorawan-stack-migrate/pkg/export:export (export device test-lux)
device_id=test-lux
correlation_id=93f84959d4f143b19a109f4024596b36
— error:pkg/rpcmiddleware/validator:field_mask_paths (forbidden path(s) in field mask)
forbidden_paths=[lora_alliance_profile_ids.vendor_profile_id lora_alliance_profile_ids.vendor_id serial_number last_seen_at]
correlation_id=f11c2dab6fd04d349d175c898b24f9a7

I have a feeling this is some sort of version mismatch between migration tool and TTS version. I can update TTS if necessary but don’t want to go through the process if it’s not needed to get this export to work.

Thanks in advance.

2

The migration tool is for migration from v2 to v3.

You can just use the standard CLI to export / import between v3 instances.

3

Thanks for the quick reply. How do I export an applications end devices using the CLI?

4

I’ve managed to use CLI’s get to set up a JSON with everything I need but it only seems to work with 1 device. Looking through the commands and flags for CLI application and end devices commands I can’t find anything to get a whole batch in one go. Is this possible?

5

Yes. Don’t specify a device, you get everything from the application, give it a try …

6

Alright, I tried the following:

ttn-lw-cli end-devices get migration-testing --name --description --lorawan-version --lorawan-phy-version --frequency-plan-id --supports-join --root-keys --mac-settings

Not specifying [device-id] returns this error as [device-id] is not an optional field for the get command:

error:cmd/ttn-lw-cli/commands:no_application_id (no application ID set)

Next I tried:

ttn-lw-cli end-devices search migration-testing --name --description --lorawan-version --lorawan-phy-version --frequency-plan-id --supports-join --root-keys --mac-settings

Which does return all devices if I don’t have any flags past description, otherwise I get

error:pkg/rpcmiddleware/validator:field_mask_paths (forbidden path(s) in field mask)
    forbidden_paths=[frequency_plan_id lorawan_phy_version lorawan_version mac_settings root_keys supports_join]
    correlation_id=d53ab07dba0b44a9afc4fa5d7a610111

Going through the documentation, everything listed in forbidden paths is a flag so I’m now confused why I’m getting this error.

What is the correct command and flags to export these devices into the correct format to be imported into another TTS instance?

7

I have a feeling this is some sort of version mismatch between migration tool and TTS version. I can update TTS if necessary but don’t want to go through the process if it’s not needed to get this export to work.

This is what I think the problem is. Version 3.17.1 is quite old as we are on 3.27.0 now. I would highly recommend migrating your setup to the newer version since we have updated certain fields and their usage may be incompatible with older TTS versions.

8

Both instances i want to import/export from are the same version though. I’m using the CLI now rather than the migration tool. I’d rather downgrade my version of the CLI, than go through a database migration and upgrade.

9

So go on then!

Makes total sense to use a matching version of the CLI - and you’d need it if you want to move from 3.17.1 to 3.27.0 via a new install of TTS.

10

Just to close out this thread, thanks for everyone’s input.

For anyone coming across this with the same issue:
1.Make sure CLI is running the same version as TTS
2. Run:

ttn-lw-cli end-devices search <application name> > devices.json

You don’t need any additional flags. This writes everything you need to import the devices to devices.json, then upload this file in the second instance and all your exported devices will be imported to this second instance.

1 Like
13

Hi there,
I’m trying to use the CLI to export devices from one instance of TTS to another instance of TTS and have come across some issues. Both versions are 3.26.0

Firstly there is no way listed in the documentation to export multiple devices at a time which seems like an oversight, the only command listed is get:

ttn-lw-cli end-devices get migration-testing test-lux --name --description --lorawan-version --lorawan-phy-version --frequency-plan-id --supports-join --root-keys --mac-settings

This works great for a singular device but cannot be used for multiple devices as the device-id field is required so get can only be used for one device at a time.
Next I tried using:

ttn-lw-cli end-devices search migration-testing

This does provide all of the devices in an application, and looking through the optional flags for search shows that all of the fields we’re after are optional flags to add to the command, however when using these flags:

ttn-lw-cli end-devices search migration-testing --name --description --lorawan-version --lorawan-phy-version --frequency-plan-id --supports-join --root-keys --mac-settings

The CLI returns this error:

error:pkg/rpcmiddleware/validator:field_mask_paths (forbidden path(s) in field mask)
    forbidden_paths=[frequency_plan_id lorawan_phy_version lorawan_version mac_settings root_keys supports_join]
    correlation_id=46c16aee396240288137d873ee5aacee

Why are these flags forbidden? I have admin rights for the TTS instance I’m logged in with, so it is not a rights issue.

Another thing I think has been missed is the payload decoder, if I use this method is there a way to retain the payload formatter for each device through the export+import process? Otherwise every individual device will have to be manually configured after import.

Anyone tackled this issue or know the correct command to get all devices per application and their necessary fields for export and import. Thanks in advance.

14

Reopened previous topic to maintain context.

This is a link to migrating to TTS which is only marginally relevant and I believe I discounted above. You. should look at the CLI docs.

Please post this as an issue on GitHub - or even better as a pull request - as this is unlikely to be spotted by anyone who can action this.

There is a general implication that devices in an application are the same - individual settings per device is really a developer convenience or for a handful of sensors.

Again, this should really be a GitHub issue to get some serious attention - it would also be appropriate to say how many devices you are thinking of to get a sense of the scale of the issue.