Data Storage Integration

Hello everyone,
I have successfully upgraded my device from V2 to V3 Stack and. I always used in V2 the Swagger to get my sensor data over HTTP.

But in V3 I am confused. Can anyone tell me how I can set up my own database to store my data locally and then retrieve that data over HTTP.

Do you have any documentation or examples?

Thank you

If not, is there a way to save only the decoded payload and ID’s in the storage API (like in V2)?

i make the HTTP Request.

curl G -H “Authorization: Bearer NNSXS.HQ3KPZW…LRFDFAI6Q7UIC2G6LT…VLMYYHZXQ”
-H “Accept: text/event-stream”
-d “limit=10”
-d “after=2021-01-20T00:00:00Z”
-d “field_mask=up.uplink_message.decoded_payload”
https://eu1.cloud.thethings.network/api/v3/as/applications/APP-ID/devices/Device-Name/packages/storage/uplink_message” \

but i got always the same error.

HTTP/1.1 501 Not Implemented
content-type: text/event-stream
referrer-policy: strict-origin-when-cross-origin
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-request-id: cafb7392-899c-47a9-bdf9-86b86e2a3652
x-xss-protection: 1; mode=block
date: Thu, 15 Apr 2021 11:12:22 GMT
content-length: 39
x-envoy-upstream-service-time: 0
server: envoy

{“code”:12,“message”:“Not Implemented”}

what i´m doing wrong?

Thank you.

Try it without the field_mask - I’m personally am still figuring the exact paths expected but I don’t have any issues with limit or date after.

At present it’s all cURL on the command line, no Swagger to play with.

Hello i remove the field_mask limit and after.

i got another Error.

{“error”:{“grpc_code”:7,“http_code”:403,“message”:“error:pkg/auth/rights:insufficient_application_rights (insufficient rights for application app-id@ttn)”,“http_status”:“Forbidden”,“details”:[{"@type":"type.googleapis.com/ttn.lorawan.v3.ErrorDetails",“namespace”:“pkg/auth/rights”,“name”:“insufficient_application_rights”,“message_format”:"insufficient rights for application {uid}",“attributes”:{“uid”:“app-id@ttn”},“correlation_id”:“374d4c08d53d4491be4e9d8472e49928”,“code”:7}]}}

Normally i want to sve the Sensor Data and retrieve wit HTTPS Request.

can you please tell me step by step how i customize my own Database to save the sensor Data and be able to retrieve this Data over HTTPS Requets?

Thanks

Sure, for your own personal database I can help. Please DM me your invoicing details …

You need to use an API key that has the permission to download data. Set Read application traffic (uplink and downlink) on. Then you should get your data.

i already used my generated API-Key to send the https-request like you see but i always got the error.

I remove the field_mask limit and after. how can i solve it?

Have you done or checked this?

yes i checkt it and is done. I retrieve the data with the API Request and i set Read application traffic (uplink and downlink).

but the request don´t work when i use the -d “field_mask=up.uplink_message.decoded_payload”

One step at a time, mostly because I can’t see your computer screen, so does it work without the field_mask?

yes it work without the filed_mask

Well as you are using the mask in the documentation, I’d say there is a bug report you need to file and hopefully it will be fixed.

But in the meanwhile at least you have something working.

Now you have to choose what your server side language of choice is and write the code to take a JSON submission and insert in to a database.

yes thank you. I have already the code to parse the JSON Data . For the insertion in to a Database that´s my point.

that´s a link that i see on the TTN website.

please look the part “Storage Integration Options” what is the Step to make this configuration.

https://www.thethingsindustries.com/docs/reference/configuration/application-server/#storage-integration-options

That is in the configuration section for the v3 stack.

TTI, not unsurprisingly, don’t provide instructions on how to write code for databases as it’s a very generic & well documented thing. Given the monumental number of other websites that provide such info, I think Google will be your friend here.

Yes, you are right, my problem is not the way I build or program my database, but I thought it was possible to connect the data received by TTS directly to my database without going through the HTTP_Request.

Huh??? What did you have in mind for the data to get from the TTN servers to your database? There is no automagic integration that puts your data in to a database.

When the data arrives at the Application Server, it will just disappear in to thin air unless YOU choose to do something with it. Those somethings include:

  • MQTT
  • WebHooks
  • Data Storage

All of which have a well documented API for your data to leave the confines of the TTN servers and get to your servers. All three provide the data as JSON with varying levels of detail. When it arrives, you parse the JSON to prepare it for however your database would like you to insert records, depending on both the database and the language you are using, perform the insert and then, erm, that’s it.