(Solved)Using TTS.DataStorage.Tab.py to retrieve data

Application Development
1

Hi - I am trying to use Nick’s python program : TTS.DataStorage.Tab.py to retrieve data from the storage integration. When I run the code I get:
Fetching from data storage …

URL: https://eu1.cloud.thethings.network/api/v3/as/applications/mydeviceid/packages/storage/uplink_message?order=received_at&type=uplink_message&after=2021-07-08T14:27:00Z
Status: 401

Traceback (most recent call last):
  File "c:/Python37/TTS.DataStorage.Tab.py", line 57, in <module>
    someJSON = json.loads(theJSON)
  File "C:\Python37\lib\json\__init__.py", line 348, in loads
    return _default_decoder.decode(s)
  File "C:\Python37\lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Python37\lib\json\decoder.py", line 353, in raw_decode
    obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting ',' delimiter: line 1 column 403 (char 402)

If I type the url above directly into a browser, then it downloads the data into the browser, no problem.

Then I added the line : print(r.text) to the code in order to see what the outcome was, and I see:

{"error":{"grpc_code":16,"http_code":401,"message":"error:pkg/identityserver:invalid_authorization (invalid authorization)","http_status":"Unauthorized","details":[{"@type":"type.googleapis.com/ttn.lorawan.v3.ErrorDetails","namespace":"pkg/identityserver","name":"invalid_authorization","message_format":"invalid authorization","correlation_id":"f8bdcc8c37814568b04afc4448d92405","code":16}]}}
Edit - Seems to be an issue with the API key?? I generated an API key and copied it, it is in the list of keys
Or is it a decoder issue?
Solved - I recreated an API key, solved the issue
Thanks
Russell

2

Yes - I think invalid_authorization is the give away :wink:

I think the problem stems from you assigning a device id to the variable called theApplication on line 3:

theApplication = "YOUR-APPLICATION-ID"

Unless of course you have an application called mydeviceid

As this won’t have the headers with the API Key, I’m a bit puzzled as to how you managed this!

3

Hi Nick - If I just type:
https://eu1.cloud.thethings.network/api/v3/as/applications/rascalmkrwan1300/packages/storage/uplink_message
I get a bunch of the uplink_message
(you can try this url as is)

4

Do you get this:

{“error”:{“grpc_code”:7,“http_code”:403,“message”:“error:pkg/auth/rights:no_application_rights (no rights for application rascalmkrwan1300@ttn)”,“http_status”:“Forbidden”,“details”:[{“@type”:“type.googleapis.com/ttn.lorawan.v3.ErrorDetails",“namespace”:“pkg/auth/rights”,“name”:“no_application_rights”,“message_format”:"no rights for application {uid}”,“attributes”:{“uid”:“rascalmkrwan1300@ttn”},“correlation_id”:“166836389d9e42268c3d14a888fc1067”,“code”:7}]}}

5

No I get -

{“result”:{“end_device_ids”:{“device_id”:“arduino-lora”,“application_ids”:{“application_id”:“rascalmkrwan1300”},“dev_eui”:“A8610A3237397A01”,“dev_addr”:“260B85F3”},“received_at”:“2021-07-08T13:24:49.179051500Z”,“uplink_message”:{“f_port”:2,“f_cnt”:1031,“frm_payload”:“MTE2LjAw”,“decoded_payload”:{“field1”:“16.00”,“field2”:49},“rx_metadata”:[{“gateway_ids”:{“gateway_id”:“3133303755006200”,“eui”:“3133303755006200”},“time”:“2021-07-08T13:24:48.856934Z”,“timestamp”:3914169411,“rssi”:-103,“channel_rssi”:-103,“snr”:-19},

(I snipped it off here)
Lots more similar info follows…

6

Is this in to a browser address bar?

Do you have some utility the allows you to set the authentication header?

Which browser?

The whole point of the API security is to stop me sitting here and ‘exploring’ your data storage or indeed any number of the API calls. So it would be interesting to know if you’ve managed to find a way around it.

7

Hi Nick, I’m simply typing the URL into my Chrome browser. Nothing more. So yes, that would seem to be a concern if anyone could do it. Perhaps TTN recognizes my pc’s IP address and authorizes the request.
As an experiment I tried the same thing using my phone to reach that URL and it rejected the request (as it should), so I think that it means I can only do it on the local machine.
I tried from Postman (on my same PC) and it returns:
{

    "code": 12,

    "message": "Not Implemented"

}

So I think we’re ok.
Further: I tried it from my Edge Browser and it didn’t authorize either. It seems like Chrome is allowing it (only from my local machine ?)
I can experiment further if need be. Although it does seem that it can only be done from my pc.
Can you simulate the issue at your end?

8

Perhaps you are logged into the V3 console with that browser?

9

Yeah, I thought of that in the first instance but no joy - I’ll have to do a browser restart as I use Safari for general and Chrome for server admin (my servers, TTN etc) so I can start with a clean slate.

10

Yes I’m no longer using the legacy console, so I’m in the new console