'GET' "decoded_payload" from uplink_message out of Storage-Integration http-Reuqest

Network and Routing The Things Stack v3
#1

Hi Community,

i’ve four sensors running in TTSv3 and Storage-Integration activated.

When i request the uplink_message for my app there is so much unnecessary data.
I just want to get the “decoded_payload” out of “uplink_message”.
Is there a way i’am missing so far?

Thanks for your time cheers

#2

You really should take the rest of the information too.

Otherwise you have no real way of knowing things like how old the packet is.

In particular, you should not be inserting timestamps or sequence numbers inside of packet bodies - because that information is already available in metadata, inserting it in the packet itself would be wasteful duplication

#3

Hi,

thank you very much for your reply! So, i’ll keep the data and parse the “for me useful” parts out of it to lower the resulting filesize.

greetings and cheers

#4

Hard disk is cheap, getting data you realise would have been useful after the fact is hard - I’d save the whole message, perhaps compacted or compressed just in case in a few weeks time you realise that something you didn’t think was useful, is.

#5

If you really know that you’ll only ever need a specific set of fields, you can tell the server to only send you those fields.

Note: This is new functionality in The Things Stack v3.11.

With the CLI that would look like this:

$ ttn-lw-cli applications storage get your-application-id \
  --type uplink_message \ # only select uplink messages
  --after-utc "2021-02-05 00:00:00" \
  --before-utc "2021-02-05 08:00:00" \
  --end-device-ids \ # you probably want device IDs
  --received-at \ # you want to know when it was received
  --up.uplink-message.f-port \ # you want the FPort
  --up.uplink-message.frm-payload \ # maybe the raw payload
  --up.uplink-message.decoded-payload # and otherwise the decoded payload

You can run ttn-lw-cli applications storage get --help to see the full list of command-line flags you can use to filter data from the storage integration.

On the HTTP API it works in a similar way:

$ curl -G "https://eu1.cloud.thethings.network/api/v3/as/applications/your-application-id/packages/storage/uplink_message" \
  -H "Accept: text/event-stream" \
  -H "Authorization: Bearer $API_KEY" \
  -d "after=2021-02-05T00:00:00Z" \
  -d "before=2021-02-05T08:00:00Z" \
  -d "field_mask=end_device_ids,received_at,up.uplink_message.f_port,up.uplink_message.frm_payload,up.uplink_message.decoded_payload"
2 Likes
#6

Hello

I tried use http request from node red but without success.
Using query from web browser as test everything is working perfect.
Can you suggest how I should setup right http request flow in node red to get storage data form Stack v3?

#7

How did you get it working in a browser - it needs the authorisation header - which may be why it’s not working so well in Node-Red.

Mostly people seem to use MQTT with Node-Red …

#8

Probably issue refer to authorization header and I tried build flow node red but without success.
I also used MQTT for “live” data but my aplication needs data logger.
Idea was not to use DB on node-red side that must be running all time but integrate with Stack v3 storage and get part of data defined via query in http request.

#9

Can you put the incoming live data in to a local database, so you aren’t reliant on the Data Storage?