Meshed - ThingSpeak Integration assistance

Hi, I’ve hit a roadblock with my first TTN implementation and would appreciate any guidance please.

I’m generally following the approach described in the published example, Collect Agricultural Data over The Things Network for passing data from my Arduino based device via TTN to ThingSpeak for visualisation. My issue is that the TTN Integration for ThingSpeak does not successfully pass the decoded payload data to my ThingSpeak channel. Some details:

I am still at proof-of-concept stage and yet to put effort into robustness etc. and the device currently sends an unchanging encoded payload. The upstream application data payload (weather observation data) is successfully decoded on TTN:

Applicn_data

using a simple decoder script:

function Decoder(bytes, port) { 
  var epochtime = ((bytes[3] << 24) | (bytes[2] << 16) | (bytes[1] << 8) | bytes[0] );
  var temp = ((bytes[5] << 8) | bytes[4])/10.0;
  var humidity = ((bytes[7] << 8) | bytes[6])/10.0;
  var baropress = ((bytes[9] << 8) | bytes[8])/10.0 + 1000.0;
  var rainfall = ((bytes[11] << 8) | bytes[10])/10.0;
  var windspeed = ((bytes[13] << 8) | bytes[12])/10.0;
  var winddirn = (bytes[15] << 8) | bytes[14];
  var lat = -38.293;
  var long = 144.996;
  var height = 105;


  return {
    field1: epochtime,
    field2: temp,
    field3: humidity,
    field4: baropress,
    field5: rainfall,
    field6: windspeed,
    field7: winddirn
  };
}

The ThingSpeak channel is in place and I have been able to directly import a file of arbitrary test data and have also used the ThingSpeak API directly to POST data into individual fields. These are shown in the following screen capture from the Channel (the data input via individual API calls is circled).

ThingSpeak_channel

My ThingSpeak integration on TTN is configured with the same write API key & channel credentials and is reported on the TTN Console as Running (v2.7.10). However, no data appears in my channel from TTN - all data in the channel is confirmed as being from my input directly to ThingSpeak.

I’m in Australia, configured on meshed-router and meshed-handler, and using the Australian console console.thethings.meshed.com.au.

With only two parameters to configure in the Integration and in the absence of error reporting, I’m at an impasse. Search of earlier Forum posts suggest this should all be straightforward now with the ThingSpeak Integration, so I’d greatly appreciate any guidance as to where I have taken a miss-step or not understood. Thanks.

1 Like

Copying @Maj in here.

(Aside, unrelated to your problems: being in Australia is no excuse to not support negative temperatures. :wink: Also, JavaScript won’t infer types when using 10.0 rather than 10. In all, temp = (bytes[5] << 24 >> 16 | bytes[4]) / 10 should do. And your epochtime, which is using 32 bits and hence for bitwise operators yields a signed integer in JavaScript, could become negative at some point. Adding a dummy zero-fill right shift >>> 0 at the end will solve that: epochtime = (bytes[3] << 24 | bytes[2] << 16 | bytes[1] << 8 | bytes[0]) >>> 0.)

Yes, the encode/decode still requires considerable refinement, which I would tackle once I’m confident the basic delivery path through the integration is demonstrable. Thanks for your suggestions.

I just noticed that 1599776952 from the screenshot is Thursday September 10th, while today it’s the 15th, at least, in Europe. :wink: Probably not an issue, but: is the screenshot old? Does ThingSpeak allow for dates in the past? Can you scroll the graphs to last week?

And above all: it is the same for all uplinks. Does ThingSpeak support specifying the very same timestamp for multiple readings?

You could use the following in the Decoder to get the server’s time:

var epochtime = Math.floor(Date.now() / 1000);

No I don’t believe this is a problem, as the field1 content (= time of observation on site, rather than time of dataset submission) is internal to the payload and could in fact represent something totally different to time. Timestamps for data submission into ThingSpeak channel would be recorded outside the payload data extracted by the TTN decoder. To illustrate this, here is the channel plot from some direct (ThingSpeak) API calls I just actioned.

In these 4 separate calls over the period 21:21:26 through 21:25:32 (GMT+10:00) I submitted

 Field 1 = 1599776952   (the same value in all 4 calls)
 Field 3 = 50.0,  51.0, 53.0, 54.0      (i.e. different data for each call)
 Field 7 = 13     (same in each call)

AdditionalPosts

These data points are correctly positioned in time, at the right hand end of the plot, and an after-the-fact export of the most recent data points from ThingSpeak verifies this:

 {"created_at":"2020-09-15T21:21:26+10:00","entry_id":38,"field1":"1599776952","field2":null,"field3":"50.0","field4":null,"field5":null,"field6":null,"field7":"13"},
 {"created_at":"2020-09-15T21:21:58+10:00","entry_id":39,"field1":"1599776952","field2":null,"field3":"51.0","field4":null,"field5":null,"field6":null,"field7":"13"},
 {"created_at":"2020-09-15T21:24:03+10:00","entry_id":40,"field1":"1599776952","field2":null,"field3":"53.0","field4":null,"field5":null,"field6":null,"field7":"13"},
 {"created_at":"2020-09-15T21:25:32+10:00","entry_id":41,"field1":"1599776952","field2":null,"field3":"54.0","field4":null,"field5":null,"field6":null,"field7":"13"}]}

Thanks for the continuing consideration of what might be astray with my integration.

1 Like

Ok, then I’m sure we need @Maj to confirm ThingSpeak should work. (And maybe peek into some logs to see what API calls are made? Yeah, that’s asking a lot…) I wonder if there’s any news on Downlink URL 502 error API + Tago + Meshed.

ThingSpeak supports times in the past if you use created_at. If you do not supply created_at, the server uses the present server time.

return {
created_at: myTime,
field1: myfield1,
field2: myField2,
etc…

You cannot use duplicate timestamps within the same channel.

I wasn’t able to test the meshed route previously, the integration works for me in US-west. Ill attempt to verify meshed success separately.

Just curious @KevinB are you using the same hardware as in the example (feather M0)?

Thanks @TriaxialNecromances for the tip on created_at which will be useful. No I am not using the Feather M0 hardware of the example - was following it just in the sense that as a newcomer, I found the write-up offered a straightforward procedural recipe for setting up the various elements of the chain.

Look forward to hearing whether the meshed route checks out OK.

Thingspeak isn’t working for me either. Also using meshed.

I’ve seen some discrepancies with the meshed server compared to the default one for US. I’ve reached out to TTN and the meshed operators, I will update here when I hear from them. @KevinB I know the guy who wrote the example and he is curious what hardware you are using, for comparison purposes.Can you share?

Hi all. Meshed has no influence or visibility on what happens in Integrations. I’ve asked TTN how they’d like to handle these kinds of requests and will provide guidance here when I receive a response.

1 Like

Thanks for the follow through on investigations.

@TriaxialNecromances I’m happy to share a few details of my project - it’s my challenge over the continuing many weeks of COVID lockdown in my city, Melbourne. I’m actually attempting to build a multisensor weather station (a well-trodden path!) informed by the 2016 write-up: Basic Weather Station. Rather than the ethernet backhaul used there, I decided to try LoRaWAN in order to reach a more suitable site for the station placement - with a senior meteorologist/climate scientist in the family, compromising the observation measurements with poor placement was never to be allowed (!) :nerd_face: The same influence expects comprehensive visualisation which is what led me to the ThingSpeak article I referenced.

I’m prototyping on a Mega2560 + Dragino LoRa Shield due to the aggregated code size of the libraries and potentially an additional interrupt. I may get to later optimisation - depends on the ease with which I might get all this working. In the meantime, it’s certainly a learning path.

The ThingSpeak Integration on the Australia cluster has been updated to the latest version. Can anyone confirm if the original problem still remains or if it’s resolved?

@Maj I’ve just now tested my ThingSpeak set up and the problem still remains - no data reaches my ThingSpeak channel. I tried deleting my existing Integration and configuring a replacement ThingSpeak Integration, without any effect. Have quadruple checked the API keys etc.

In the meantime I have been able to try a parallel integration with Tago and data is being passed from TTN to my Tago device bucket successfully.

Update: The ThingSpeak integration is now working correctly for me. This does not coincide with any changes I can observe in my set up. It ‘came to life’ and device data started being delivered into myThingSpeak channel at 2020-09-25 19:00:45 GMT+10:00.

Thanks to all who assisted with investigating my issue.

Yes, I think it’s finally resolved with an update from the TTN Core Team late last week.