Hi everyone,
Is there any limitation to get the data stored through the Storage Integration? I mean any restriction with the amount of data or something like that. I have been storing some sensor data during 5 days and when I try to get the data with query?last=5d it just returns 1 and a half days of data. On the other hand if I try with query?last=2d it returns 1 and a half days of data too. So, it seems like a limitation related to the amount of data I am trying to get from the database. The data is stored in the server but it´s being truncated.
Can someone help me to clarify this?
I’m not seeing an restrictions. Getting all data† gets me a bit over 1,500 readings for my most active device.
If you’re getting fewer results than that, then something odd is going on.
If you get the same number of results, or more, for a single device for just 1.5 days, then you’re exceeding the Fair Access Policy, which for the smallest payload in the best circumstances (1 byte on SF7) allows for at most 647 messages per day for EU868. Though that Fair Access Policy is not being enforced yet, maybe the Data Storage Integration implicitly limits storage to, say, 4,500 messages per week?
(Please don’t repost the same question when getting no answer, but edit the previous one to add more details. I’ve deleted your earlier post about the same thing.)
† The Data Storage integration keeps data for 7 days. It seems the data is truncated at midnight, so using ?last=7d gets me data as of March 10, shortly after midnight, being 7.5 days worth of data.
Thank you so much for your answer @arjanvanb. I will check the amount of data I am storing in 1 day. The strange thing is I am able to get the data if I change the period of time I can check the data is there.
So how many items are returned when querying the integration? (A count for occurences of }, will quickly give you a number.)
What do you mean by this? Ah, I see different details in your deletedhidden post. Is it related to not getting the most recent data? Emphasis mine:
When I try e.g. to download data related to the 3 last days I am getting the information from the first and second day and then just a few data from the present day. On the other hand, if I download just the last day of data, the data is properly downloaded. So, I know the data is being properly stored but there is some problem when I try to get it.
So, the following returns different 1.5 days worth of data?
I am afraid I am exceding the amount of data permited by the Fair Access Policy cause a bad operation of my sensor. So I will do some change in my code to reduce this amount of data.
Any way, I have checked again and all the data is being stored. The problem is when downloading it.
Yes, when the query is last=5 it returns e.g from 13/03 9:00 to 14/03 23:00, the rest of data is truncated;and when the query is last=4 it returns e.g from 14/03 9:00 to 15/03 23:00. So the data truncated in the first case appears in the second case.
Is there some other query apart from last? Can I download just the 5th day ago of data?
The number of items is variable because the application send the variation of a measurement just when it overpasses a pre-stablished threshold. For one day the number of results is near 1000 but with the proper operation of the sensor it should be less than half of this.