# How to use Storage Integration for device data?

**URL:** <https://www.thethingsnetwork.org/forum/t/how-to-use-storage-integration-for-device-data/47350>\
**Category:** V3 Integrations\
**Created:** [May 3, 2021, 2:21pm UTC](https://www.thethingsnetwork.org/forum/t/how-to-use-storage-integration-for-device-data/47350 "2021-05-03T14:21:40Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Kampi](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/kampi/32/35697_2.png) [@Kampi](https://www.thethingsnetwork.org/forum/u/Kampi)\
**Post date:** [May 3, 2021, 2:21pm UTC](https://www.thethingsnetwork.org/forum/t/how-to-use-storage-integration-for-device-data/47350/1 "2021-05-03T14:21:41Z")

</div>

I have an application with a LoRaWAN device and I want to use the storage integration to store the data. The storage integration is enabled by opening Integrations → Storage Integration → Enable and I can simulate live data which gets stored.

![image](https://www.thethingsnetwork.org/forum/uploads/default/original/3X/3/2/32a421926b10967f93ea330299035c008a8252bc.png)

I can see the data by using the API, but the data from the device aren´t stored in the database.

How can I forward my device data into my database to store them?

---

<div class="post-metadata">

**Author:** ![descartes](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/descartes/32/47220_2.png) [@descartes](https://www.thethingsnetwork.org/forum/u/descartes)\
**Post date:** [May 3, 2021, 2:37pm UTC](https://www.thethingsnetwork.org/forum/t/how-to-use-storage-integration-for-device-data/47350/2 "2021-05-03T14:37:24Z")

</div>

> [@Kampi](#):
>
> I can see the data by using the API, but the data from the device aren´t stored in the database.

Which API and which database are you referring to?

> [@Kampi](#):
>
> How can I forward my device data into my database to store them?

Data Storage you fetch the data, other integrations forward.

MQTT or WebHooks give you the maximum amount of information. MQTT you need an application running to listen for messages & process them. WebHooks you can write a web server script to receive the data & process them.

Data Storage you request the data for an application or just a device & process them.

---

<div class="post-metadata">

**Author:** ![Kampi](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/kampi/32/35697_2.png) [@Kampi](https://www.thethingsnetwork.org/forum/u/Kampi)\
**Post date:** [May 3, 2021, 3:03pm UTC](https://www.thethingsnetwork.org/forum/t/how-to-use-storage-integration-for-device-data/47350/3 "2021-05-03T15:03:41Z")

</div>

Hi @descartes,

I´m referring to this database

![image](https://www.thethingsnetwork.org/forum/uploads/default/original/3X/7/7/775a7d8a8673d290eae089c2093b6af0221b4eb6.png)

I thought that the storage integration will store the data in my account and then I can access them as I wish.

---

<div class="post-metadata">

**Author:** ![descartes](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/descartes/32/47220_2.png) [@descartes](https://www.thethingsnetwork.org/forum/u/descartes)\
**Post date:** [May 3, 2021, 3:08pm UTC](https://www.thethingsnetwork.org/forum/t/how-to-use-storage-integration-for-device-data/47350/4 "2021-05-03T15:08:32Z")

</div>

And you can access it as you wish - not only is it all detailed in the documentation but there are even some sample web addresses being displayed on the console for you.

> [@Kampi](#):
>
> I can see the data by using the API, but the data from the device aren´t stored in the database.

What do you mean when you say you can see your data by using the API?

How do you know the data is not being stored in the database?

If you can see your data, surely that means it is stored in the database!

---

<div class="post-metadata">

**Author:** ![Kampi](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/kampi/32/35697_2.png) [@Kampi](https://www.thethingsnetwork.org/forum/u/Kampi)\
**Post date:** [May 3, 2021, 3:12pm UTC](https://www.thethingsnetwork.org/forum/t/how-to-use-storage-integration-for-device-data/47350/5 "2021-05-03T15:12:59Z")

</div>

Hi,

I use the following Python snipped to access the data:

```
import json
import requests

Application = "kampi-smartmetering"
APIKey = ""
Fields = "up.uplink_message.decoded_payload,up.uplink_message.frm_payload"
NumberOfRecords = 30
URL = "https://eu1.cloud.thethings.network/api/v3/as/applications/" + Application + "/packages/storage/uplink_message?order=-received_at&limit=" + str(NumberOfRecords) + "&field_mask=" + Fields
Header = { "Accept": "text/event-stream", "Authorization": "Bearer " + APIKey }

print("\n\nFetching from data storage...\n")

r = requests.get(URL, headers = Header)
JSON = "{\"data\": [" + r.text.replace("\n\n", ",")[:-1] + "]}";

print("URL: {}\n".format(r.url))
print("Status: {}\n".format(r.status_code))
print("Response: {}\n".format(r.text))
print("JSON: ")
print(json.dumps(json.loads(JSON), indent = 4))
print()

```

The script only returns the data which I have simulated (4 data points) over the `Messaging` tab

![image](https://www.thethingsnetwork.org/forum/uploads/default/original/3X/5/6/563732d0bc6852144de5a383e62f65d220ad855d.png)

but not the data from the device over the last hours (18 messages).

---

<div class="post-metadata">

**Author:** ![descartes](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/descartes/32/47220_2.png) [@descartes](https://www.thethingsnetwork.org/forum/u/descartes)\
**Post date:** [May 3, 2021, 3:53pm UTC](https://www.thethingsnetwork.org/forum/t/how-to-use-storage-integration-for-device-data/47350/6 "2021-05-03T15:53:38Z")

</div>

Somewhat frustrating that you just didn’t start with this as that’s either my code or you are a miracle mind-reader!

My code is working. Your code is working with my application & key.

I don’t know how long is storing uplinks in Data Storage in v3. Can you trigger an uplink and then try?

Have you tried removing the `limit` just in case?

---

<div class="post-metadata">

**Author:** ![Kampi](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/kampi/32/35697_2.png) [@Kampi](https://www.thethingsnetwork.org/forum/u/Kampi)\
**Post date:** [May 3, 2021, 4:26pm UTC](https://www.thethingsnetwork.org/forum/t/how-to-use-storage-integration-for-device-data/47350/7 "2021-05-03T16:26:02Z")

</div>

Hi,

removing `limit` doesn´t change something. My device is sending new data every hour. The simulated data are older than the last data from my device.  
But I have configured a payload formatter in my device. May this formatter cause this issue?

---

<div class="post-metadata">

**Author:** ![descartes](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/descartes/32/47220_2.png) [@descartes](https://www.thethingsnetwork.org/forum/u/descartes)\
**Post date:** [May 3, 2021, 4:41pm UTC](https://www.thethingsnetwork.org/forum/t/how-to-use-storage-integration-for-device-data/47350/8 "2021-05-03T16:41:03Z")

</div>

> [@Kampi](#):
>
> May this formatter cause this issue?

If it is generating an issue I’d still expect the raw part of the payload to be stored.

I’d try removing the sort and the field filters, again, worth a try.

Then I’d try turning off the payload formatter, again just to see.

---

<div class="post-metadata">

**Author:** ![kersing](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/kersing/32/44777_2.png) [@kersing](https://www.thethingsnetwork.org/forum/u/kersing)\
**Post date:** [May 3, 2021, 5:13pm UTC](https://www.thethingsnetwork.org/forum/t/how-to-use-storage-integration-for-device-data/47350/9 "2021-05-03T17:13:37Z")

</div>

> [@Kampi](#):
>
> May this formatter cause this issue?

Yes it can. What happens if you reset to the default formatted?

---

<div class="post-metadata">

**Author:** ![Kampi](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/kampi/32/35697_2.png) [@Kampi](https://www.thethingsnetwork.org/forum/u/Kampi)\
**Post date:** [May 3, 2021, 5:24pm UTC](https://www.thethingsnetwork.org/forum/t/how-to-use-storage-integration-for-device-data/47350/10 "2021-05-03T17:24:49Z")

</div>

I will check it.

Edit: I have checked it and it seems that the formatter was the issue. The data is stored correctly after I removed it.

---

<div class="post-metadata">

**Author:** ![cetinkayaoktay](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/cetinkayaoktay/32/39117_2.png) [@cetinkayaoktay](https://www.thethingsnetwork.org/forum/u/cetinkayaoktay)\
**Post date:** [August 3, 2021, 11:22am UTC](https://www.thethingsnetwork.org/forum/t/how-to-use-storage-integration-for-device-data/47350/11 "2021-08-03T11:22:07Z")

</div>

Hi,

I am having the same issue and removing the formatter didn’t work.

I believe the v3 stores and shows the incoming data but I couldn’t manage to access the data that I took last week.

Do you have any suggestions?

Thank you!

---

<div class="post-metadata">

**Author:** ![descartes](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/descartes/32/47220_2.png) [@descartes](https://www.thethingsnetwork.org/forum/u/descartes)\
**Post date:** [August 3, 2021, 11:40am UTC](https://www.thethingsnetwork.org/forum/t/how-to-use-storage-integration-for-device-data/47350/12 "2021-08-03T11:40:08Z")

</div>

> [@cetinkayaoktay](#):
>
> I couldn’t manage to access the data that I took last week.

What have you tried? What are you expecting in the way of accessing data?

---

<div class="post-metadata">

**Author:** ![cetinkayaoktay](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/cetinkayaoktay/32/39117_2.png) [@cetinkayaoktay](https://www.thethingsnetwork.org/forum/u/cetinkayaoktay)\
**Post date:** [August 3, 2021, 11:58am UTC](https://www.thethingsnetwork.org/forum/t/how-to-use-storage-integration-for-device-data/47350/13 "2021-08-03T11:58:47Z")

</div>

I took some measurements last week, and I was able to see the data on the console and through the sample web addresses provided at the storage integration tab when I was taking the measurements. However, I didn’t save them, and there is nothing on the console/web address now.

I tried the above code, but it didn’t return anything. When I use my other applications reporting frequently, it shows only the real-time data that you can see through the web address, but there is no historical data, e.g., two days or a week old.

I need what I measured; raw or processed doesn’t matter.

---

<div class="post-metadata">

**Author:** ![descartes](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/descartes/32/47220_2.png) [@descartes](https://www.thethingsnetwork.org/forum/u/descartes)\
**Post date:** [August 3, 2021, 12:28pm UTC](https://www.thethingsnetwork.org/forum/t/how-to-use-storage-integration-for-device-data/47350/14 "2021-08-03T12:28:23Z")

</div>

The Data Storage only holds data for up to about 36 hours.

You may want to look at these for ways of exporting Data Storage with a cron job or using MQTT or Webhooks:

> **[GitHub - descartes/TheThingsStack-Integration-Starters: Starter / Template code...](https://github.com/descartes/TheThingsStack-Integration-Starters)**
>
> Starter / Template code for various The Things Stack (v3) integrations - GitHub - descartes/TheThingsStack-Integration-Starters: Starter / Template code for various The Things Stack (v3) integrations

---

<div class="post-metadata">

**Author:** ![cetinkayaoktay](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/cetinkayaoktay/32/39117_2.png) [@cetinkayaoktay](https://www.thethingsnetwork.org/forum/u/cetinkayaoktay)\
**Post date:** [August 3, 2021, 1:32pm UTC](https://www.thethingsnetwork.org/forum/t/how-to-use-storage-integration-for-device-data/47350/15 "2021-08-03T13:32:32Z")

</div>

Okay, thank you. I am checking it now.

Just to confirm, it won’t be possible to get those data back, right?

---

<div class="post-metadata">

**Author:** ![descartes](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/descartes/32/47220_2.png) [@descartes](https://www.thethingsnetwork.org/forum/u/descartes)\
**Post date:** [August 3, 2021, 2:13pm UTC](https://www.thethingsnetwork.org/forum/t/how-to-use-storage-integration-for-device-data/47350/16 "2021-08-03T14:13:16Z")

</div>

Correct, if it’s not there, it’s gone.

It’s not a primary data source for the stack, it’s there mostly if your other integration(s) go down you can grab the lost data from there.

My backup Webhook is on a £7/year server, so some PHP web space shouldn’t cost much and doesn’t even have to be in country.

This shows setup:

[![](https://www.thethingsnetwork.org/forum/uploads/default/original/3X/f/7/f7be18066c475f65c4f39ac0b9431495f5970752.jpeg "Getting Started with Webhooks - Nick McCloud (descartes)") ](https://www.youtube.com/watch?v=yMXlTDljz_E)

---

<div class="post-metadata">

**Author:** ![cetinkayaoktay](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/cetinkayaoktay/32/39117_2.png) [@cetinkayaoktay](https://www.thethingsnetwork.org/forum/u/cetinkayaoktay)\
**Post date:** [August 3, 2021, 2:52pm UTC](https://www.thethingsnetwork.org/forum/t/how-to-use-storage-integration-for-device-data/47350/17 "2021-08-03T14:52:53Z")

</div>

Thank you for your help. I will surely do that.

---

<div class="post-metadata">

**Author:** ![safe-99](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/safe-99/32/35697_2.png) [@safe-99](https://www.thethingsnetwork.org/forum/u/safe-99)\
**Post date:** [January 30, 2022, 11:57am UTC](https://www.thethingsnetwork.org/forum/t/how-to-use-storage-integration-for-device-data/47350/18 "2022-01-30T11:57:51Z")

</div>

is there a example for javascript ?

---

<div class="post-metadata">

**Author:** ![descartes](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/descartes/32/47220_2.png) [@descartes](https://www.thethingsnetwork.org/forum/u/descartes)\
**Post date:** [January 30, 2022, 12:19pm UTC](https://www.thethingsnetwork.org/forum/t/how-to-use-storage-integration-for-device-data/47350/19 "2022-01-30T12:19:51Z")

</div>

What do you want to do?

More details other than just the Python being translated in to JavaScript please.

---

<div class="post-metadata">

**Author:** ![safe-99](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/safe-99/32/35697_2.png) [@safe-99](https://www.thethingsnetwork.org/forum/u/safe-99)\
**Post date:** [January 30, 2022, 12:28pm UTC](https://www.thethingsnetwork.org/forum/t/how-to-use-storage-integration-for-device-data/47350/20 "2022-01-30T12:28:32Z")

</div>

hi i get this Data but how I can fetch it with the api key in javascript ? I don´t know how i use the api key to make a authorized

[https://eu1.cloud.thethings.network/api/v3/as/applications/jjjj/devices/eui-70b3d57ed004bf93/packages/storage/uplink\_message](https://eu1.cloud.thethings.network/api/v3/as/applications/jjjj/devices/eui-70b3d57ed004bf93/packages/storage/uplink_message)

the result :

{“result”:{“end\_device\_ids”:{“device\_id”:“eui-70b3d57ed004bf93”,“application\_ids”:{“application\_id”:“jjjj”},“dev\_eui”:“70B3D57ED004BF93”,“dev\_addr”:“260BA217”},“received\_at”:“2022-01-29T21:18:17.682195763Z”,“uplink\_message”:{“f\_port”:1,“frm\_payload”:"/////wXDAAAADQACAAU=",“decoded\_payload”:{“Etage”:2,“Gebaeude”:13,“Lichtintensität”:1475,“Luftfeuchtigkeit”:65535,“Raum”:5,“Reed-Kontakt(1=Geschlossen, 0=Geoeffnet)”:0,“Temperatur”:65535},“rx\_metadata”:[{“gateway\_ids”:{“gateway\_id”:“lorawangatewayy”,“eui”:“58A0CBFFFE8047F5”},“time”:“2022-01-29T21:18:17.416009902Z”,“timestamp”:2478936795,“rssi”:-54,“channel\_rssi”:-54,“snr”:10.25,“location”:{“latitude”:54.34513235377525,“longitude”:10.092231893285023,“source”:“SOURCE\_REGISTRY”}}],“settings”:{“data\_rate”:{“lora”:{“bandwidth”:125000,“spreading\_factor”:7}},“coding\_rate”:“4/5”,“frequency”:“868100000”,“timestamp”:2478936795,“time”:“2022-01-29T21:18:17.416009902Z”},“received\_at”:“2022-01-29T21:18:17.476084832Z”,“consumed\_airtime”:“0.066816s”,“network\_ids”:{“net\_id”:“000013”,“tenant\_id”:“ttn”,“cluster\_id”:“ttn-eu1”}}}}

[Next page](https://www.thethingsnetwork.org/forum/t/how-to-use-storage-integration-for-device-data/47350.md?page=2)