Discussion point: V3 console is too complicated

TTN V2 to V3 Upgrade V3 Console
1

An interesting point that came up during a community session at one of the conferences is that the V3 console is too complicated. It prevents adoption from newcomers, but even for veterans like myself it is very difficult getting all the settings correct when registering a new device. And now we’re not even talking about migrating an old device. Debugging is also much more difficult due to too many log output lines.

Coft, Staging and V2 were great because of their simplicity, and that’s likely the reason for the fast adoption of TTN the past 5 years.

What are your opinions? Perhaps we can make a list of suggestions for TTI to consider.

17 Likes
2

I’ve been using Chirpstack for the last 3 years and the user experience is remarkably more simple on that platform than v3.

I still don’t understand why I need to have one application for each payload encoder even though it’s unlikely that I’ll be deploying exactly the same devices to each project - most projects will include agritech sensors, weather stations, air quality monitors etc., all of which come from different manufacturers with different codecs so then instead of having one application per “customer” or project, I need to have at least three applications per customer/project. Chirpstack allows you to select the decoder at the device level, so I can have a single project with devices from multiple manufacturers in it, all being decoded correctly.

The “auto add” of devices is a great idea, but there don’t seem to be that many devices out there - certainly for Tektelic there are a significant number of their sensors missing so you have to go down the manual route.

FWIW, I prefer the three-step process of registering a device but again, coming from chirpstack where it’s a single form, this is definitely less straight forward than other offerings out there.

Edit: I should probably add here that v3 is significantly better from a user experience perspective than the Loriot console, so it’s not all bad… :smiley:

3 Likes
3

Good topic for discussion. Some of the complexity of setting up a new device ‘should’ be helped and mitigated by the use of the ‘Device Repositary’ for off the shelf products (but that is still limited and often required knowledge of specific hardware or firmware versions that may not be available or obvious to users), which should improve over time as more devices added. For makers and new product developers or hackers then you are faced with the issue raised. Another key point is GW and Application/Device traffic seen live is often useful for debugging and understanding what is going on and seems simpler in V2, however, V3 is extremely chatty and verbose when looking at live data and needs better filters and perhaps colour que’s - remember orange flash for join req or the green flash - easy to spot and pick out in passing traffic even on busy GW’s but difficult to spot on V3 live. Some nodes see many GW’s in their area (not just the development GW on the bench) and so traffic can flow thick and fast and with multiple duplicate entries per receiving GW this can be a 'mare… some nodes I know see up to 12 GW’s in close proximity. So UI experience around V3 console needs consideration and filtering options (ironically making for more complexity, options and buttons!)

EDIT: For any TTI core team members who read this thread remember:

  1. We are not beating up on you!
  2. We are not the enemy but are here to help and trying to make TTN/TTS/TTI better for all!
  3. This is to be a mechanism for reasoned positive contributions and explainations for why something might be wrong or open for improvement - not (just) to dump on you guys :wink:
4 Likes
4

You can, if the device has a decoder - now called a formatter - in the repository that is linked in the back ground for automagical use.

You can set a payload formatter for the application or use the repository.

You can set a payload formatter for a device, or use the application one or use the repository.

I can’t think of a combination that won’t work.

There is a minor butt in all this. Due to some code bloat from some users (200KB!!) that are already seeing timeouts on payload formatting processing (set to 100ms), there will be a restriction on the size of the code to 4KB: Reduce payload formatter script size · Issue #4053 · TheThingsNetwork/lorawan-stack · GitHub

This timeout can occur even if you have very sleek & efficient code - if the JS engine is under pressure due to a whole pile of heavy weight formatters or just a lot to do at that point in time, it can still timeout. In other words, payload formatters are convenience functions not to be relied upon.

To that end I only use the payload formatters for development & testing - for production I decode at my end, that way I can be assured of up to date decoding and if the servers are under pressure, I get my fields anyway.

6 Likes
5

Ah, ok, thanks, that’s definitely not clear from the UI - perhaps I’ve missed the documentation on that one!

Edit: Actually, looking at it it’s not quite the same.

In Chirpstack, you have the concept of “device profiles”. These device profiles are created at “organisation” level and contain the decoders.

You then have multiple applications within your organisation which can have multiple devices, and when you add a device you select the device profile and it automatically adds the appropriate decoder.

I guess this is a bit like how the repo side of things is supposed to work, but you don’t need to get code merged to a repo in order for your data to be decoded.

8

Please invite them to add more of their their sensors to the open source database. There are two available so there is a start…

9

Interesting that Tektelic was just a sponsor and presenter at todays conf… but limited engagement with this enabling facility! :thinking:

2 Likes
10

Hours of fun to be had figuring out One UI to Rule Them All and I would prefer an organisation to have its own set of profiles for different elements.

Creating a custom UI that includes such functionality is entirely feasible.

11

I’ve already got the decoders for these particular sensors, I’m assuming I can contribute to the database via a pull request or similar?

I’ve been out of touch on this for a while! :slight_smile:

2 Likes
12

Yes, very much appreciated. The repo is at GitHub - TheThingsNetwork/lorawan-devices: Device Repository for LoRaWAN devices

1 Like
13

Thanks, I’ve put in a PR for the Agri-sensor this morning, I’ll move questions about that to another topic so we can keep the conversation about v3 going here… :slight_smile:

2 Likes
14

First of all, I’m generally happy with the v3 console.

Some of the things I don’t really like:

  • In the live view, for each incoming message I see at least 6 lines being logged. Some of them contain exactly the same information and this makes it a bit hard to follow.
  • In the end device view, it shows the creation date/time as a column, this is mostly irrelevant for me. Much more useful would be “last seen time” or something like that.
  • In the live view, time stamps are sometimes incorrect (I think this happens between 0:00 local time and 0:00 UTC)

I really hate the fact that I can no longer see through which gateway a message arrived, but that is not the console’s fault.

1 Like
15

At the moment I have two gateways running at nearly the same position. One V2 and one V3. My V3-node is received by both gateways so I get two “receive data messages”, one of them is discarded. The gateway-message coming from the V3-gateway contains gateway ID and EUI of my V3-gateway, the one coming from my V2-gateway says “packetbroker”.
imho this is quite normal for the moment because V2 is bridged to V3 and the bridge is the packet-broker.

16

Not particularly responding to @bertrik, more information for all.

The database is split in to “stuff I need to know immediately”, which is cached in Redis, something the TTI crew call Hot and stuff I really can’t afford to keep up to date just because.

So the Hot Stuff is mostly about uplink processing and all the other network server shenanigans. The console just creeps in with some access to the hot data, but overall, unless you are looking at an actual detail screen, a lot is cached on the console server, periodically updated.

All that said, I too would much prefer the created TS changed to Last Seen. If that can’t be done, as I find it irritating information to my OCD brain, I’d prefer the column wasn’t there.

Apart from the Live Data tab which is a server event stream which can’t be easily replicated due to web browser security model (CORS), pretty much everything can be redone. I don’t think it’s a huge hardship not having the fire hose of detail that the v3 console gives, as mostly on a day to day level we just want the v2 level of information. The challenge there is that to grab any form of traffic console data, we need to either have a proxy for MQTT to our web browser as MQTT via web sockets is not enabled (at present) or use a WebHook (HTTP Integration) to a server that then pushes incoming data down to a browser. The MQTT proxy is problematic because it would need a users credentials and web sockets is an additional burden on the TTI servers.

If this magic console was a desktop app, then it becomes a little simpler as we can use the API and MQTT to provide a management interface with real time status. As soon as you (me) start creating an application you have support issues (won’t run on Windows 3.1 with only 640KB of RAM - can’t think why not, 640KB should be enough for anybody). So this option isn’t floating my boat.

  1. Let’s not pick at the details of the how the v3 stack is architected or if I didn’t quite get it 101% right.
  2. If peeps could use the forum search more, I’d have more time for my console-lite, code name “JM’s little helper”.

If you do want to respond to this post, tell me the things you most need in a console-lite, in reverse order of importance, using the numbered bullet point formatting. For bonus points, put some fun wording in, even native, with a link to YouTube on it.

1 Like
17

I guess near the top of the list then would be a filtered version of the firehose pipe of hot stuff/live with a tick box option to only show unique/deduplicated messages vs the full flow. Best done in the console so people without the neccessary skill-sets dont have to worry about how to set up http integration/webhooks or mtqtt pipes to extract just what they are interested in. 2nd to that would then be some way of perhaps colour coding as mentioned above? to make certain messages/types stand out.

Word has it that JM’s slow vs OCD brain might better cope with what is seen and rumour has it he doesnt like drinking from firehoses but rather would see filtered cooled vs hot streams sipped through a straw… or atleast well chilled and in measured quantities https://www.youtube.com/watch?v=uWVQwIDjtNY

18

Well done that man! Not only not formatted correctly, but totally missed the point about not being able to access the live data without using MQTT or a WebHook.

But research progresses at this end, so all is not lost. But the minimum will be snagging the Personal API Key so that it can possibly then setup an MQTT feed.

Pretty colours easy once we’ve got a console going.

19

But do I get extra bonus point for JM = John Mills?! :wink:

Will go take a walk to think about needs… (English woodland not N.African desert)

(Will) Delete surpless message once read to keep feed clean!

20

Sorry, yeah, JM for the win.

Head full of potential, causing some degree of angst at the state of the software development universe.

You can delete this and the three above later but don’t lose the YouTube link

21

Hi,

I have made for myself (but usable by others) a little console to see only some information a quick way:
https://ngraziano.github.io/alternate-ttn-console/devices source: GitHub - ngraziano/alternate-ttn-console

To add live data I was thinking about using the events API (Working with Events | The Things Stack for LoRaWAN) and filter only the message of type “as.up.data.forward” to get the field decoded_payload and some network information to get the same information that go to MQTT but maybe i am wrong.

But as always day job is getting all my time so maybe one day I have time to try.

2 Likes
22

Cool!

I’ve something similar that uses Data Storage as an early hack a few months back but the code base isn’t publishable due to some technical & NDA issues. But I’m working on it.

That will work with the CLI but won’t work in a browser due to the use of event stream as mentioned above.

Looking at the API or the CLI (http://descartes.co.uk/TTSv3CLI.html) we can’t create Pub/Sub or WebHooks so some setup will be required with any other streams.

1 Like