TTN V2 to The Things Stack V3 Transition

Sure:

My applications and devices exist as a series of datapoints in the “old” system.

I expect that I can either transfer or copy these datapoints to the new console or I expect a download (zip file) that I then might upload into the v3 console, simple as that.

I expect an information about the steps I have to take (first copy the data and then delete the old data - or download the old data, delete it and then upload into the new system.

Honestly, I don’t like the idea to install a software to do this: These are just data points. Please provide format conversion etc. as download / export from the v2 system.

I tried the “ttnctl-darwin-amd64.zip” for Mac, is unzips to a 19,5 MB binary, In my books, script files are much smaller. I have unzipped it now 4 times and only once it looked like a shell script.

Why don’t you install the shell script on your machines to provide downloadable JSON for everyone?

Andreas

Are app/join EUIs no-longer assigned from the TTN block in V3?

Hate to keep raising problems @johan - guess @Maj will think me a ‘wingeing Pom’ to borrow a cricketing term! - but playing devil’s advocate here and trying to concieve solutions/mitigation strategies… The user may not have control of the GWs or own them, also simply migrating the GW’s to V3 if they do have control simply switches off other V2 users nodes/applications covered by them - not very community spirited :frowning:

:thinking: How can we co-ordinate these activities and transitions. Some users - especially many end users - may not even be aware of pending transition and behind the scenes activity and just receive a service or the output from a dashboard integration perhaps on a shared url link… then suddenly one day it stops working/updating…

in @esairq 's case above clearly he didnt get the telegram saying do x before doing y, and likely is only the 1st of many - we need to quickly come up with a very simple (so people like me can do it!) step by step guide and publish it professionally and well signposted or the community is going to get very fragmented & very messy very quickly (and the Forum likely wont be a nice place to be for a while! :scream: )

The moderators I’m sure will curate and clean up and guide where they can but this could get overwhelming I fear…

From what I am getting on back channels 3 main issues look to be biggest concern - timing with RO from April even if actual sunset much later - everyone dealing with pandemic and thinking this is too much too quickly. And 2nd is the lack of V2 to V3 dataflow atleast during transition phase…may not be technically possible or a big lift to fix but can something be done before chaos lets loose. Many now getting the transition message and looking far out to sea the Tsunami is already building :wink: And that leads to 3rd concern that GW’s that are relied on may switch before users ready/have time/can access nodes to effect change or update…many indeed wont even know if what they have in their hands or have deployed in the field falls into (which) one of the 4 categories of device you called out depending if build or bought of the shelf and if a screwdiver build or s/w compiled themselves etc. They simply may not know where to start!

Whilst we cant cover all the self builds and hacks people have put together & deployed on TTN perhaps we can post a traffic light system for popular off the shelf systems - Green - will transition with no issue, Amber may need a reset (OTA or power cycle) or a minor intervention, some ttnctl cmd line hacking or whatever, Red - oops you’ve got a potential problem/may be stuffed and need to work hard! And apply that to the likes of say (just for examples) Laird RS1xx T&H, Dragino LHT65, ERS-xyz, MCF88-abc, DecentLabs-ghi etc…(Maybe start with labeling the items listed in the TTN/TTI Marketplace?!) That way we can focus efforts and manage by exception/scale of problem…? :thinking: Perhaps this is something we can recruit the product manufacturers/suppliers into doing?

5 Likes

@johan

The instructions in Migrating from The Things Network Stack V2 are incomplete and not all clear. Please provide the missing information and update the instructions.

  • Instructions appear to be written for migration to thethingsindustries.com.
    There is no mentioning of The Things Network or thethings.network URLs.

  • Configure V2 CLI

    3. Create a file called .ttnctl.yml in your home directory

    For Windows users make sure to use the --config flag with the full path to the configuration file: ttnctl.exe --config <fullpath\>/config.yml
    …”

    1. The title says “configure V2 CLI” but the config file appears to contain V3 related settings.

    2. Why use a different config file name for Windows? The different name is confusing (only the leading dot should be removed for Windows).

    3. The contents specified for .ttnctl.yml will not work for Windows.
      Windows requires backslash (not forward slash) as path separator and the default user folder of a Windows user is C:\Users\<user_name>.

    4. What is folder /home/<user_name>/.ttnctl used for?
      Storing data that is internally used by the ttnctl program?

      I ran ttnctl on Windows (without first creating a config file due to incomplete instructions). As a result a .ttnctl folder was automatically created in my current folder, which can actually be any folder. This is probably not the preferred way.

    5. One needs to specify <domain_id> in several places.
      Nowhere is mentioned what value must be used for <domain_id>.

    6. Can we simply use URL’s like [*.]<domain_id>.thethings.industries and only need to use some <domain_id> specific for The Things Network V3, or do we need completely different URL’s?
      This is unclear because the (eu) URL for The Things Network V3 Console is eu1.cloud.thethings.network/console which does not match any of the <domain_id>.thethings.industries URLs specified for the config file.

    7. The config file mentions “handler-id: <domain_id>-handler”.
      Is this a V3 handler-id? “Configure V2 CLI” gives the impression it should be a V2 handler and the IDs generated by the command ttnctl discover handler are very different.

2 Likes

Hi @johan, I will be pending when the deployment of nam1 is ready, to start the migration of the devices and gateways here on Colombia, could you explain the effect of RX delay change on the operation of the devices?

I have managed to deploy the RAK7258 gateway on V3. It is showing as connected but no live data is shown when in the backend of the RAK7258 I can see incoming data.

This is indeed a real issue. The reason why all gateway coverage of V2 is available in V3, is really to remove the immediate need to reconfigure gateways. There is no reason now to migrate gateways.

If we need to clarify this further, I’m happy to do so, and I’d like to know in which way you suggest.

This is a good idea. We welcome a page for this on the documentation pages. This is open source, see GitHub - TheThingsIndustries/lorawan-stack-docs: Documentation for The Things Stack.

Thanks. I filed an issue for this, see Clarify V2 to V3 migration guide · Issue #210 · TheThingsIndustries/lorawan-stack-docs · GitHub.

The increased RX1 delay will make the downlink path more reliable. If gateways have a high latency backhaul or if you want to have application-layer downlink responses in the RX1 or RX2 window, this will now most likely all work. When devices join The Things Network V3, they will automatically get the new RX1 delay.

Please file issue at GitHub - TheThingsNetwork/lorawan-stack: The Things Stack, an Open Source LoRaWAN Network Server

1 Like

@Johan, could you provide an estimate on when au1 and nam1 clusters will be available?

Thank you, I am able to login today.

1 Like

I just migrated a device (RAK7200 GPS tracker) with LPP data format from v2 to v3 stack.
Unfortunately I cannot find the myDevices(Cayenne) integration.

How can I get this integration in v3?

BR,

Jeroen

Many of our community members are no techies, so they will have many trouble with using a non gui tool like ttnctl. As suggestion, is it possible to make an simple export function in the old V2 console? So the user can click on a few buttons and became as result the json of the choosen devices for importing them in V3.

1 Like

They’re scheduled for early February. We’re gonna closely watch the eu1 cluster next week and tag a 3.11 release, so we can go ahead with the other clusters.

myDevices unfortunately does not support the V3 webhook format. These are minimal changes. I’ll contact them again to add support.

It’s easier than ever to add these templates: GitHub - TheThingsNetwork/lorawan-webhook-templates. It’s just that the remote platform needs to support the JSON structure.

It’s a good suggestion, but the V2 Console is really EOL.

We might still find away to add an “import” screen in the V3 Console where you can enter your V2 application ID, access key and potentially list of devices.

FYI we’re internally discussing the feasibility to add an application and device import feature in the V3 Console.

8 Likes

Ok, that would be easier. The login data is the same when login in V3 as in V2, in that case after the login, the system identifies the user and his applications and devices in V3 and V2 too. With this, it is not possible to list the devices from V2 in the “import” screen in the V3 Console?

Ooops, thats new - and a bit of a show stopper…one of the most used Integrations I suspect and certainly one of my go to’s for a quick demo to potential clients or to show Community users the art of the possible/benefits of LoRaWAN & TTN…

4 Likes

True, it is almost mandatory…
Helium supports myDevices

2 Likes

Gateway Privacy Settings

In V2:

capture 2021-01-28 15·33·47

In V3:

capture 2021-01-28 15·31·29

and

capture 2021-01-28 15·44·54

Questions

  • Why is ‘Gateway status Public’ default disabled in V3?

  • Why is ‘Publish location’ default disabled in V3?

  • Has option ‘Make owner public’ been removed?
    Is owner now private or public?

Status and location need to be public to make the gateway visible on maps so others can check if they are within range of network coverage.

Owner by default should be private.

1 Like

For TTN community connected GW’s default should be status & location public, though I can understand that with e.g EU GDPR owner may now have to be opt-in vs opt out (but not neccessarily in other parts of the world?) - though sure should be there to opt in to as an enabler for community communication and info exchange and also to request help in debugging and checking coverage etc…

Also as a Community Initiator where I see new GW’s pop up in any of the Communities I have set up I often reach out to the owner and invite them to join the local TTN community, if not already a member, as part of the process - if I dont see who the owner is that initiative of Community building breaks! :slight_smile:

If status and location are not enabled by default, many (new) users will just not be aware or leave them disabled (‘just because’).

GDPR may play a role here but user is not clearly and explicitly informed about these choices. This will need to be improved in the console.

(Otherwise) we may end up with very empty gateway maps after the migration.

1 Like

Well that’s great for Helium but that’s completely irrelevant.

Like I said, myDevices doesn’t support The Things Stack V3, and I will notify them again on the importance of being able to process data from The Things Stack V3.

We should probably make this enabled by default for public networks, like The Things Network, indeed. I filed an issue: Make default for public gateway status, location and contact info configurable · Issue #3712 · TheThingsNetwork/lorawan-stack · GitHub

There’s really no such thing as an owner. This was already the case with V2, but that was a bit hacky. There are collaborators only, and those can be users or organizations.

So we have two things now:

  1. Collaborators: users and orgs that have access
  2. Contact information: name, email and phone, that can be public or private

The third thing we may want to add here is the owner, or the sponsor in terms of TTN, which could be a user or an org.

What do you think @kschiffer ?

Gateway location and status is not considered personal data. We may, however, want to add some “circle” and round the exact coordinates, so that it is not exactly clear on which roof the gateway is installed. But I don’t see legal limitations here.

Indeed this is a use case that we should keep supporting. I included this in the issue I linked above.

1 Like