Hello,
I’m trying to migrate all my ABP end devices to V3, but there is a problem.
The ttn-lw-migrate tool tells me: permission denied: V2 clusters are now read-only when it tries to clear the keys from the V2 console.
This is also refrenced in this post: TTN v2 API down
Leaving the keys on V2 will give routing conflicts and deleting the application in V2 supposedly does not delete the devices.
Is there any way to get around this?
Greetings G
If you edit (e.g.swap) the last two bytes of the AppKey the V2 implementation will stop functioning and V3 versions will take over the role. For a time the fact V2 was set RO scuppered any change to the key limiting this ability, but this has been re-enabled over the last week or so
Yeah, but disabling the migration tool is just perverse!
As @htdvisser seems to be away, perhaps @kschiffer could comment.
Thanks for your response.
I tried editing the last two bytes of the App Session Key of a device, but i get the same error.
I can delete the individual devices, but i dont know if this prevents the routing conflicts. Does anybody know if deleting a device stops the V2 implementation form working?
Deletes/edits should always be allowed. If the migration tool/and or API is not working as intended, that’s a bug and we didn’t do that intentionally. I’ll check that.
We’ve identified the issue and a fix is on the way.
That’s great news! Thank you for the quick response.
We just deployed a patch that should fix the issue. Please inform us here how it goes.
Hello, I am still getting the error when drying to export applications or devices using ttnv2 as the source.
error:unknown:unknown (permission denied: V2 clusters are now read-only. Migrate your devices to a The Things Stack (V3) cluster now! V2 clusters will permanently shut down in December 2021.)
What command are you running?
export TTNV2_APP_ID="my_application_name"
export TTNV2_APP_ACCESS_KEY="ttn-account-v2.mykeyishere"
export FREQUENCY_PLAN_ID="US_902_928_FSB_2"
ttn-lw-migrate application --source ttnv2 "my_application_name" > application_export.json
The key and application name are replaced but these are basically what I’m using, from:
Github
Can confirm. The export still doesn’t work using command
ttn-lw-migrate application --source ttnv2 “application name” --verbose > devices.json
Ok but can you paste the verbose outputs? I just tested with one of my applications and I didn’t have any issues
~ » ttn-lw-migrate application --source ttnv2 "myapp" --verbose > apps.json
DEBUG [core]parsed scheme: ""
DEBUG [core]scheme "" not registered, fallback to default scheme
DEBUG [core]ccResolverWrapper: sending update to cc: {[{discovery.thethings.network:1900 <nil> 0 <nil>}] <nil> <nil>}
DEBUG [core]ClientConn switching balancer to "pick_first"
DEBUG [core]Channel switches to new LB policy "pick_first"
DEBUG [core]Subchannel Connectivity change to CONNECTING
DEBUG [core]Subchannel picks a new address "discovery.thethings.network:1900" to connect
DEBUG [core]pickfirstBalancer: UpdateSubConnState: 0xc000847750, {CONNECTING <nil>}
DEBUG [core]Channel Connectivity change to CONNECTING
DEBUG [core]Subchannel Connectivity change to READY
DEBUG [core]pickfirstBalancer: UpdateSubConnState: 0xc000847750, {READY <nil>}
DEBUG [core]Channel Connectivity change to READY
DEBUG rpc-client: call done auth-type=key duration=282.012642ms method=/discovery.Discovery/GetByAppID service-name=ttn-lw-migrate service-version=2.x.x
DEBUG [core]parsed scheme: ""
DEBUG [core]scheme "" not registered, fallback to default scheme
DEBUG [core]ccResolverWrapper: sending update to cc: {[{eu.thethings.network:1904 <nil> 0 <nil>}] <nil> <nil>}
DEBUG [core]ClientConn switching balancer to "pick_first"
DEBUG [core]Channel switches to new LB policy "pick_first"
DEBUG [core]Subchannel Connectivity change to CONNECTING
DEBUG [core]Subchannel picks a new address "eu.thethings.network:1904" to connect
DEBUG [core]pickfirstBalancer: UpdateSubConnState: 0xc000848670, {CONNECTING <nil>}
DEBUG [core]Channel Connectivity change to CONNECTING
DEBUG [core]Subchannel Connectivity change to READY
DEBUG [core]pickfirstBalancer: UpdateSubConnState: 0xc000848670, {READY <nil>}
DEBUG [core]Channel Connectivity change to READY
DEBUG rpc-client: call done auth-type=key duration=120.676701ms method=/handler.ApplicationManager/GetDevicesForApplication service-name=ttn-lw-migrate service-version=2.x.x
DEBUG rpc-client: call done auth-type=key duration=117.271201ms method=/handler.ApplicationManager/GetDevice service-name=ttn-lw-migrate service-version=2.x.x
INFO Clearing device keys dev_eui=xxxxx device_id=xxxx
DEBUG rpc-client: call done auth-type=key duration=131.202206ms method=/handler.ApplicationManager/SetDevice service-name=ttn-lw-migrate service-version=2.x.x
DEBUG [core]Channel Connectivity change to SHUTDOWN
DEBUG [core]Subchannel Connectivity change to SHUTDOWN
DEBUG [core]Channel Connectivity change to SHUTDOWN
DEBUG [core]Subchannel Connectivity change to SHUTDOWN
DEBUG [transport]transport: loopyWriter.run returning. connection error: desc = "transport is closing"%
Sure here is the verbose output:
The first device is succesfull because i have already cleared the keys of this device a while ago.
DEBUG [core]parsed scheme: “”
DEBUG [core]scheme “” not registered, fallback to default scheme
DEBUG [core]ccResolverWrapper: sending update to cc: {[{discovery.thethings.network:1900 0 }] }
DEBUG [core]ClientConn switching balancer to “pick_first”
DEBUG [core]Channel switches to new LB policy “pick_first”
DEBUG [core]Subchannel Connectivity change to CONNECTING
DEBUG [core]Subchannel picks a new address “discovery.thethings.network:1900” to connect
DEBUG [core]pickfirstBalancer: UpdateSubConnState: 0xc000976620, {CONNECTING }
DEBUG [core]Channel Connectivity change to CONNECTING
DEBUG [core]Subchannel Connectivity change to READY
DEBUG [core]pickfirstBalancer: UpdateSubConnState: 0xc000976620, {READY }
DEBUG [core]Channel Connectivity change to READY
DEBUG rpc-client: call done auth-type=key duration=260.536037ms method=/discovery.Discovery/GetByAppID service-name=ttn-lw-migrate service-version=2.x.x
DEBUG [core]parsed scheme: “”
DEBUG [core]scheme “” not registered, fallback to default scheme
DEBUG [core]ccResolverWrapper: sending update to cc: {[{eu.thethings.network:1904 0 }] }
DEBUG [core]ClientConn switching balancer to “pick_first”
DEBUG [core]Channel switches to new LB policy “pick_first”
DEBUG [core]Subchannel Connectivity change to CONNECTING
DEBUG [core]Subchannel picks a new address “eu.thethings.network:1904” to connect
DEBUG [core]pickfirstBalancer: UpdateSubConnState: 0xc000885e40, {CONNECTING }
DEBUG [core]Channel Connectivity change to CONNECTING
DEBUG [core]Subchannel Connectivity change to READY
DEBUG [core]pickfirstBalancer: UpdateSubConnState: 0xc000885e40, {READY }
DEBUG [core]Channel Connectivity change to READY
DEBUG rpc-client: call done auth-type=key duration=1.969800231s method=/handler.ApplicationManager/GetDevicesForApplication service-name=ttn-lw-migrate service-version=2.x.x
DEBUG rpc-client: call done auth-type=key duration=54.222061ms method=/handler.ApplicationManager/GetDevice service-name=ttn-lw-migrate service-version=2.x.x
INFO Clearing device keys dev_eui=1000000000000001 device_id=1000000000000001
DEBUG rpc-client: call done auth-type=key duration=55.772106ms method=/handler.ApplicationManager/SetDevice service-name=ttn-lw-migrate service-version=2.x.x
DEBUG rpc-client: call done auth-type=key duration=50.553326ms method=/handler.ApplicationManager/GetDevice service-name=ttn-lw-migrate service-version=2.x.x
INFO Clearing device keys dev_eui=1000000000000002 device_id=1000000000000002
DEBUG rpc-client: call failed auth-type=key duration=49.663047ms error=rpc error: code = PermissionDenied desc = permission denied: V2 clusters are now read-only. Migrate your devices to a The Things Stack (V3) cluster now! V2 clusters will permanently shut down in December 2021. method=/handler.ApplicationManager/SetDevice service-name=ttn-lw-migrate service-version=2.x.x
DEBUG [core]Channel Connectivity change to SHUTDOWN
DEBUG [core]Subchannel Connectivity change to SHUTDOWN
DEBUG [core]Channel Connectivity change to SHUTDOWN
DEBUG [core]Subchannel Connectivity change to SHUTDOWN
DEBUG [transport]transport: loopyWriter.run returning. connection error: desc = “transport is closing”
Error: error:go.thethings.network/lorawan-stack-migrate/cmd:export (export device 1000000000000002)
DEBUG [transport]transport: loopyWriter.run returning. connection error: desc = “transport is closing”
error:go.thethings.network/lorawan-stack-migrate/cmd:export (export device 1000000000000002)
device_id=1000000000000002
— error:unknown:unknown (permission denied: V2 clusters are now read-only. Migrate your devices to a The Things Stack (V3) cluster now! V2 clusters will permanently shut down in December 2021.)
DEBUG [core]parsed scheme: ""
DEBUG [core]scheme "" not registered, fallback to default scheme
DEBUG [core]ccResolverWrapper: sending update to cc: {[{discovery.thethings.network:1900 <nil> 0 <nil>}] <nil> <nil>}
DEBUG [core]ClientConn switching balancer to "pick_first"
DEBUG [core]Channel switches to new LB policy "pick_first"
DEBUG [core]Subchannel Connectivity change to CONNECTING
DEBUG [core]Subchannel picks a new address "discovery.thethings.network:1900" to connect
DEBUG [core]pickfirstBalancer: UpdateSubConnState: 0xc00079f320, {CONNECTING <nil>}
DEBUG [core]Channel Connectivity change to CONNECTING
DEBUG [core]Subchannel Connectivity change to READY
DEBUG [core]pickfirstBalancer: UpdateSubConnState: 0xc00079f320, {READY <nil>}
DEBUG [core]Channel Connectivity change to READY
DEBUG rpc-client: call done auth-type=key duration=90.184182ms method=/discovery.Discovery/GetByAppID service-name=ttn-lw-migrate service-version=2.x.x
DEBUG [core]parsed scheme: ""
DEBUG [core]scheme "" not registered, fallback to default scheme
DEBUG [core]ccResolverWrapper: sending update to cc: {[{us-west.thethings.network:1904 <nil> 0 <nil>}] <nil> <nil>}
DEBUG [core]ClientConn switching balancer to "pick_first"
DEBUG [core]Channel switches to new LB policy "pick_first"
DEBUG [core]Subchannel Connectivity change to CONNECTING
DEBUG [core]Subchannel picks a new address "us-west.thethings.network:1904" to connect
DEBUG [core]pickfirstBalancer: UpdateSubConnState: 0xc00087c330, {CONNECTING <nil>}
DEBUG [core]Channel Connectivity change to CONNECTING
DEBUG [core]Subchannel Connectivity change to READY
DEBUG [core]pickfirstBalancer: UpdateSubConnState: 0xc00087c330, {READY <nil>}
DEBUG [core]Channel Connectivity change to READY
DEBUG rpc-client: call done auth-type=key duration=403.674152ms method=/handler.ApplicationManager/GetDevicesForApplication service-name=ttn-lw-migrate service-version=2.x.x
DEBUG rpc-client: call done auth-type=key duration=99.549464ms method=/handler.ApplicationManager/GetDevice service-name=ttn-lw-migrate service-version=2.x.x
INFO Clearing device keys dev_eui=xxxx device_id=xxxx
DEBUG rpc-client: call failed auth-type=key duration=107.868155ms error=rpc error: code = PermissionDenied desc = permission denied: V2 clusters are now read-only. Migrate your devices to a The Things Stack (V3) cluster now! V2 clusters will permanently shut down in December 2021. method=/handler.ApplicationManager/SetDevice service-name=ttn-lw-migrate service-version=2.x.x
DEBUG [core]Channel Connectivity change to SHUTDOWN
DEBUG [core]Subchannel Connectivity change to SHUTDOWN
DEBUG [core]Channel Connectivity change to SHUTDOWN
DEBUG [core]Subchannel Connectivity change to SHUTDOWN
Error: error:go.thethings.network/lorawan-stack-migrate/cmd:export (export device `xxxx`)
error:go.thethings.network/lorawan-stack-migrate/cmd:export (export device `xxxx`)
device_id=xxxx
--- error:unknown:unknown (permission denied: V2 clusters are now read-only. Migrate your devices to a The Things Stack (V3) cluster now! V2 clusters will permanently shut down in December 2021.)Ok I can reproduce this. I’ll get back here after some debugging
Ok I’ve pushed a fix. My tests work. Can you guys check now?
I’ve tested exporting both applications and devices and they both appear to be working. Thank you!
It seems, that for migrating an application it only works for the first device. Here the output.
DEBUG [core]parsed scheme: ""
DEBUG [core]scheme "" not registered, fallback to default scheme
DEBUG [core]ccResolverWrapper: sending update to cc: {[{discovery.thethings.network:1900 <nil> 0 <nil>}] <nil> <nil>}
DEBUG [core]ClientConn switching balancer to "pick_first"
DEBUG [core]Channel switches to new LB policy "pick_first"
DEBUG [core]Subchannel Connectivity change to CONNECTING
DEBUG [core]Subchannel picks a new address "discovery.thethings.network:1900" to connect
DEBUG [core]pickfirstBalancer: UpdateSubConnState: 0x14000b499c0, {CONNECTING <nil>}
DEBUG [core]Channel Connectivity change to CONNECTING
DEBUG [core]Subchannel Connectivity change to READY
DEBUG [core]pickfirstBalancer: UpdateSubConnState: 0x14000b499c0, {READY <nil>}
DEBUG [core]Channel Connectivity change to READY
DEBUG rpc-client: call done auth-type=key duration=85.612125ms method=/discovery.Discovery/GetByAppID service-name=ttn-lw-migrate service-version=2.x.x
DEBUG [core]parsed scheme: ""
DEBUG [core]scheme "" not registered, fallback to default scheme
DEBUG [core]ccResolverWrapper: sending update to cc: {[{eu.thethings.network:1904 <nil> 0 <nil>}] <nil> <nil>}
DEBUG [core]ClientConn switching balancer to "pick_first"
DEBUG [core]Channel switches to new LB policy "pick_first"
DEBUG [core]Subchannel Connectivity change to CONNECTING
DEBUG [core]Subchannel picks a new address "eu.thethings.network:1904" to connect
DEBUG [core]pickfirstBalancer: UpdateSubConnState: 0x14000760770, {CONNECTING <nil>}
DEBUG [core]Channel Connectivity change to CONNECTING
DEBUG [core]Subchannel Connectivity change to READY
DEBUG [core]pickfirstBalancer: UpdateSubConnState: 0x14000760770, {READY <nil>}
DEBUG [core]Channel Connectivity change to READY
DEBUG rpc-client: call done auth-type=key duration=38.450209ms method=/handler.ApplicationManager/GetDevicesForApplication service-name=ttn-lw-migrate service-version=2.x.x
DEBUG rpc-client: call done auth-type=key duration=40.95725ms method=/handler.ApplicationManager/GetDevice service-name=ttn-lw-migrate service-version=2.x.x
INFO Clearing device keys dev_eui=10521C5AD67CFEFF device_id=pax_heltecv2_10521c5ad67cfeff
DEBUG rpc-client: call done auth-type=key duration=41.688584ms method=/handler.ApplicationManager/SetDevice service-name=ttn-lw-migrate service-version=2.x.x
DEBUG rpc-client: call done auth-type=key duration=40.17475ms method=/handler.ApplicationManager/GetDevice service-name=ttn-lw-migrate service-version=2.x.x
INFO Clearing device keys dev_eui=10521C5AD70CFEFF device_id=pax_heltecv2_10521c5ad70cfeff
DEBUG rpc-client: call failed auth-type=key duration=35.519958ms error=rpc error: code = PermissionDenied desc = permission denied: V2 clusters are now read-only. Migrate your devices to a The Things Stack (V3) cluster now! V2 clusters will permanently shut down in December 2021. method=/handler.ApplicationManager/SetDevice service-name=ttn-lw-migrate service-version=2.x.x
DEBUG [core]Channel Connectivity change to SHUTDOWN
DEBUG [core]Subchannel Connectivity change to SHUTDOWN
DEBUG [core]Channel Connectivity change to SHUTDOWN
DEBUG [core]Subchannel Connectivity change to SHUTDOWN
Error: error:go.thethings.network/lorawan-stack-migrate/cmd:export (export device `pax_heltecv2_10521c5ad70cfeff`)
error:go.thethings.network/lorawan-stack-migrate/cmd:export (export device `pax_heltecv2_10521c5ad70cfeff`)
device_id=pax_heltecv2_10521c5ad70cfeff
--- error:unknown:unknown (permission denied: V2 clusters are now read-only. Migrate your devices to a The Things Stack (V3) cluster now! V2 clusters will permanently shut down in December 2021.)Welcome, it would be very useful to know what command you issued to get this debug log.
Please note, I used the </> tool to format it - please do the same in future.