Feather M0 with LMIC takes more than 2 minutes to transmit

See arduino-lmic/src/hal/hal.cpp at master · mcci-catena/arduino-lmic · GitHub but I would also check the code for any other loops that call hal_ticks() or hal_checkTimer().

To do this, you’d also have to fixup the result of the above routines so that they reflect the time elapsed during sleep (but you’re going to have to do that for any solution that doesn’t completely remove and replace the LMiC schedular).

And you’d need to have your event method schedule your next transmission job for a future time using the LMiC scheduler.

But as hinted at before, you should probably start by studying

Issues · mcci-catena/arduino-lmic · GitHub

And maybe try words like “suspend” too. Make sure not to search with the isopen criteria…

Hmmmm, key difference here is that you have told the LMIC to schedule the task to run at some point in the future which is up to LMIC. And if it’s been frozen in time, it may take some time to get to a point where it thinks it’s time to send.

Whereas I tell the LMIC to transmit, which as long as you haven’t contravened it’s algorithm for duty limits, will do a transmit via it’s own internal threaded system. And, as you can see from the table below, it’s reasonably consistent.

I’d set LMIC_DEBUG_LEVEL to 1 in config.h in the library so you can see far more detail on the serial port as to what is going on under the hood.

Here’s the last 10 records from the Production office temperature device. The code has to turn on the IO & ADC, then each sensors is turned on, some settle time, a reading (or more to iron out noise) and then turn off. One sensor is a DHT and the other is a DS18B20, both of which have to be re-initialised. There’s also a 4 second extra as a rounding error on the set interval of 300 seconds. It’s not optimised and I’ve never timed it as I’ve no particular need for an Arduino + RFM95 node as I don’t deploy them for production, just as Canarys (end to end monitor) or as give-aways.

Timestamp Difference Diff from Ave
03/12/2020 19:40:00 5m 34s 526ms -0m 0s 652ms
03/12/2020 19:45:34 5m 34s 730ms -0m 0s 856ms
03/12/2020 19:51:08 5m 33s 389ms 0m 0s 485ms
03/12/2020 19:56:41 5m 32s 876ms 0m 0s 998ms
03/12/2020 20:02:14 5m 33s 6ms 0m 0s 868ms
03/12/2020 20:07:48 5m 34s 15ms -0m 0s 141ms
03/12/2020 20:13:22 5m 34s 563ms -0m 0s 689ms
03/12/2020 20:18:56 5m 33s 403ms 0m 0s 470ms
03/12/2020 20:24:30 5m 34s 231ms -0m 0s 357ms
03/12/2020 20:30:04 5m 34s 1ms -0m 0s 127ms

Given I’ve never really looked at my Arduino implementation at this level of detail, I’m reasonably happy but my interest is peaked so I’ll instrument a copy of the device at some point over the holidays.

Not sure what function you’re using to do that (apparently not LMIC_setTxData2() which is the only official method) but anyway, if the entire time between the last transmission and this request is an anesthesia type “unaware of time passing” sleep, then that request will indeed violate limits.

Maso93’s transmissions are occurring at an interval which is the sum of the duration of the “lost awareness” sleep, plus LMiC’s idea of reasonable pacing as the passage of non-sleep time it is aware of.

Fixing up LMiC’s sense of elapsed time is pretty critical - LMiC needs to see that low power sleep time as interval between packets, rather than only start measuring interval after waking up.

Thanks for your insights; for what I’ve understood now I have two options to solve this LMiC timing issue:

  • somehow fix the library code in order to make it aware of the sleeping time - for which I may have found a solution inside this issue published on the now deprecated LMiC library mantained by Matthijs Kooijman (hoping it will be compatible with the LMiC library I’m currently using)
  • understand which clock is used by the LMiC library to keep track of time and how to save it - for

Anyway I didn’t fully understand this part:

You are implying that it would be better to use the os_setTimedCallback function? If yes, why? I saw that function used on the examples provided in the library’s repo, but I think that even if I managed to correct the timing issue it would be better not to rely on the LMiC library for the transmission scheduling but “impose” it using LMIC_setTxData2(); also I’m afraid that the eventual correction would probably lead to some imprecisions in the LMiC clock.

Many have tried this, for instance:

and many others to find by various searches on the forum.

I may have just realized what’s the difference you’re talking about in terms of the transmission scheduling: in mine, I execute the LMIC_setTxData2() inside the do_send function, while in yours you execute it directly in a function you defined.

I haven’t really understood how the do_send function works, in particular with respect to its parameter (osjob_t* j), but I think that using it may somehow leave to the LMiC library the decision on when to send the transmission with all its implications; whereas directly calling the LMIC_setTxData2() as you did maybe will enforce the library to actually send asap, without taking into account the timing and other parameters that may cause issues like the one I’m experiencing.

Regarding possible duty cycle violations I don’t think I should be in danger of violating them, since the payload I’m sending each hour is really small (10 bytes) and using the TTN airtime calculator the airtime should range from 60 to 1500 ms, in any case way below the 1% duty cycle established for most of the bands (which should correspond to a maximum of 36 seconds per hour).

Unfortunately I won’t be able to work on this project until monday, but thanks to the advices you and @cslorabox gave me I have plenty of tests to prepare and run on my Feathers!

Anyway thanks again for sharing your insights and the results of your work, which by the way is quite impressive!

(Also thanks for the last link you sent, I only noticed it now and I’ll keep it in mind for the tests I’ll do next monday)

The LMIC has a mini-task scheduler built-in - not a RTOS but an element of one. Internally it creates, starts, stops and deletes jobs that the schedule will then process according to the jobs criteria.

They create two jobs for the user - one for data collection and one for sending. The data structure is osjob_t* and that is used by sendjob but it could be called anything because it’s only a placeholder to attach the next scheduled time and the function to call, which is do_send. This requires the job structure as a parameter as that’s how the scheduler calls it, as the structure can hold other information if need be.

All that said, calling do_send should be similar to me calling LMIC_setTXData2 but with some initial checks so it doesn’t try whilst it is already processing. My state flags tell me it won’t be, as my structure is either sleeping, creating a payload or LMIC active.

But I don’t have visibility on all your code so I can’t be sure what else is going on - for instance, I don’t even define the ‘sendjob’.

All of this has to be in the context of me trying many many things summer 2019, including trying to adjust timings and so forth. I stripped out various elements from the main example, had debug of the LMIC on full and read the source code. I arrived at a stable solution that works that I’m happy I’ve not warped time & space to achieve, pragmatic but not ideal. Read the README.md for my archive to get a better picture.

At present for co-developer (usually juniors), leveraging Arduino libraries, I use Uno/Nano/Pro Mini/Nano Every with a RAK4200 module on AT commands. I have done a project using a Nano Every (48K Flash, 6K RAM) with an RFM95 using LMIC to good effect. I’m trying out the Adafruit Feather M0 with RFM95 as this is a first class citizen of the MCCI LMIC repro but like many other of these projects, they tend to stop at the basic examples - your requirement for sleep is very common but it’s not been included as there are other things for the developers to do and it’s possibly a bit application specific.

I’ve been to “Planet Change LMIC Timing” and lost days of my life as I detailed on my repro. I’ve learn’t much since then but I’m not tempted back, I’d rather waste my time re-writing a LoRaWAN stack for smaller devices. I was highlighting that with a forum search you can find other people with similar issues over the years. Question is, do you feel lucky (punk), well do you?

It’s critical to remember that this only attempts to stage a packet and set state variables.

it does not send anything!

That only happens if and when LMiC decides it’s appropriate to during a future execution of the runloop.

Typically you wouldn’t call this until you were ready to send, but the actual sending is still going to happen sometime later which takes minimum intervals into account.

The reason one would want to use LMiC’s scheduler to plan the transmission for a future time is so that there can be one scheduling mechanism, which you can then modify so that if it has nothing to do for a while it sleeps.

Otherwise you end up with two scheduling mechanisms that aren’t well aware of each other and both of which need to be made compatible with the fact that at least one of them (the custom one) can cause a sleep which effects both.

Just to update, I tried various solutions without luck:

  • using the code mentioned above, trying both the “classic” do_send function and a custom function (essentially identical but without the osjob_t parameter), with the loop containing the os_runloop_once() function and with different values for the sleep_subperiod, ranging from 5 seconds to few minutes for each “subsleep” with the total sleep time always amounting to 1 hour; this resulted in even longer time periods between each transmission, especially in the case of lower subperiods;

  • following the solution illustrated in this link, which basically consisted in adding the following lines of code to the hal.cpp file of the LMiC library:

    u4_t os_cumulated_sleep_time_in_seconds=0L;

    u4_t hal_ticks () {
    static uint8_t overflow = 0;
    uint32_t scaled = (micros()+os_cumulated_sleep_time_in_seconds*1000000) >> US_PER_OSTICK_EXPONENT;

    ]

    void hal_sleep_lowpower (u1_t sleepval) {
    os_cumulated_sleep_time_in_seconds += sleepval;
    }

This last solution resulted in transmission periods with seemingly randomic lenghts, but again often longer than 2 minutes.

At this point, I’m thinking of downgrading the LMiC library to a prior version, since it seems like before the most recent updates it was more “loose” in its scheduling routines.

I’ll keep you (and everyone else reading this post :slight_smile: ) updated.

1 Like

I’ve found my actual Feather (think archeological dig in my workspace) - I’ll implement a sleep routine to see what happens.

Using the MCCI LMIC straight out the packet I can totally trash the duty cycle / fair use policy - like 20 seconds - which indicates the comment-free code in that section isn’t working as expected but it’s very hard to figure out why (or indeed what it’s mean’t to do, and lacking any design docs, what it was intended to do).

Any duty cycle enforcement would seem to be here:

This is really a case where it would make sense to add some logging; unfortunately that’s a bit harder in a .c source file in an Arduino library, but I believe there’s a helper method that could be called.

Specifically what you’d want to track would be txbeg getting set to anything other than 0 or LMIC.txend in the processing of this “do we want to transmit” clause.

Particularly you’d want to study anytime this test fails:

I got as far as this:

Indeed, LMIC.bands[].avail tracks the next allowed transmit per “band” - more on that at the end

It’s set here, along with the global duty cycle version

So what is a “band” ? Well, it seems to be a way to track various duty cycle limits on various sub-channel groups.

This mention seems to have the most explanation though I think the actual values in use may be coming from calls elsewhere to LMIC_setupBand()

Given this logic is required I’m not going to get too deep into defeating it; if one were always going to sleep for hours sure, but many things have stay-awake phases, too.

You know what would be really simple? Sleep for 57 minutes and 40 seconds.

Really, thank you for your commitment in searching that deep into the libraries, unfortunately I’m not very skilled in the great field that is the firmware development; I’ll try my best to understand the lines of code both of you mentioned but I’m not really optimistic about the results that I’ll get by doing that.

Yesterday I tried to load my script using a downgraded version of LMiC (2.3.2), but again with no luck :confused:
Anyway I found out that the actual problem may be in the TX_COMPLETE event firing: I tried a script with a TX period of 100 seconds and it seemed that after the first 2-3 executions, where the TX period overhead was acceptable (< 30 seconds), the Feather woke up from the deep sleep, sent its message and only then stayed on for another few minutes, probably waiting for the TX_COMPLETE event to fire, since it’s where the sleep flag is set; I derived this information just from listening to the USB port connection and disconnection sounds, since this time the Feather was still connected to my laptop during the script execution, and by observing the messages related to transmissions in the Telegram channel I’m using for debugging.

Believe me, I’d love to solve this issue with a simple fix like that one :sweat_smile: that would work in the case my only concern was to have a more precise TX period, but actually my problem is that if the node stays on for more than 2 minutes, instead of the few seconds that many report with the Feather M0, its battery life will be drastically affected, and sadly that’s a key aspect of my project.

As this is the heart of the matter, probably worth writing the exact timings down - what you see in the serial port (the Arduino IDE can add timestamps) against the uplink timestamp - there should be no reason for TX_COMPLETE to not follow on ~2.5 seconds after the uplink - so something is going wrong if it isn’t.

If you can post all your code (without the keys/EUI’s), I can try it here.

I’d really appreciate it, thanks!
In order to replicate my configuration you should connect the IO1 pin to pin 6; regarding the sensors they shouldn’t have anything to do with my problem and even if you don’t connect anything to the board the firmware should handle their absence by sending some special values.
I had to rename the files to fit the format restrictions imposed by the forum, obviously you’ll need to remove the final “.txt” :sweat_smile:

Firmware_FeatherM0.ino.txt (9.1 KB) myHeader.h.txt (2.1 KB)

1 Like

Hi @descartes, did you manage to try the code on your Feather M0?

Yes, I’ll have to dig it out and re-run the tests as I don’t appear to have recorded the results anywhere.

Ok perfect, I’ll keep an eye on this post for the results then! Thanks for your time :slight_smile: