What LMIC state must be preserved during sleep mode?

[bluejedi]

Thanks. This may serve as an example.
My goal is to be able to use non-volatile memory of arbitrary type, not just RTC memory (which has a limited lifetime for the number of writes per cell) and not only for ESP32.

Getting and setting state is something that should be implemented in the LMIC library itself. Any state implementation details and its intricasies should be encapsulated from clients that are using the library.

The responsibility of the application would be to save and restore the state to/from volatile memory. The type of volatile memory used is a client dependent implementation detail and not the responsibility of the LMIC library.

The ESP32 specific LMIC fork mentioned uses ESP32 RTC memory but that is limited to both ESP32 and its RTC memory. From design perspective the saving and restoring of state to/from (whatever form of) non-volatile memory should not be done in the LMIC library.

[EricVdB]

FRAM will be your friend here

[bluejedi]

FRAM is one of the non-volatile memory types I want to be able to use it with.

[grazy]

@bluejedi I agree that the save should be made in lmic library, and should adapt to different kind of storage. Maybe it can be made by a function which call a callback in user program to save on or more block of byte.

To be generic I think that all the value modified by OTAA join response and by mac command should be save.
It includes, session keys but also RX2 configuration, channels configuration, current datarate, … (like describe in the link put by @arjanvanb)

But for the state to be correct the time must also be correct (for globalDutyavail and Band.avail for global and band duty rate) or maybe you can ignore theses values if you sleep long enough (maybe more than 3 hours, I did not do the exact calculation).

[bluejedi]

@grazy

Maybe it can be made by a function which call a callback in user program to save on or more block of byte.

While a callback is a nice option I think it is not required here and possibly makes things more complex / dependent / slower.

Time is a good point. Many/most nodes will not have a realtime clock nor can they access a (NTP) time server, so I’m curious how the time issue might be solved or (reliably) worked around in this case.

Maybe @htdvisser has some ideas for this.

[arjanvanb (Arjan)]

As an aside, I once noticed the following:

@arjanvanb

When a counter increments, it seems LMiC allows for storing it in some “device database” (beware: my own terminology). Like for LoRaWAN’s FCntUp :

LMIC.seqnoUp += 1;
DO_DEVDB(LMIC.seqnoUp,seqnoUp);

The same macro is used after Over-the-Air Activation, or for adaptive date rate. However, DO_DEVDB is a no-operation in oslmic.h :

#define DO_DEVDB(field1,field2) /**/

(Which, I think, is also why this code even compiles, as the bare seqnoUp is not even defined?)

But maybe my guess about the meaning of the DO_DEVDB macro is wrong. First, for a database, I’d have expected LMiC to define that in the HAL, not in oslmic.h . Also, I don’t see how the values are ever read back from such database, in the IBM code.

[EricVdB]

@grazy

But for the state to be correct the time must also be correct (for globalDutyavail and Band.avail for global and band duty rate) or maybe you can ignore theses values if you sleep long enough (maybe more than 3 hours, I did not do the exact calculation).

In case of an ESP32, the RTC keeps running in deep sleep, so I guess time is not an issue here, or did i misunderstood the issue?

[bluejedi]

I think you are correct. Preserving ‘time’ is probably not an issue there, only when the ESP32 is powered off.

[arjanvanb (Arjan)]

…or does it

@matthijs

the fact that micros() does not update might even be the cause of your problem in the first place: you queue a new packet after the duty cycle delay is over, but because you’ve been sleeping, LMIC doesn’t know that time has passed. Ideally, the micros() counter would be adapted to compensate for the sleep time which requires some changing some global variable internal to the Arduino code, but that might not be so portable. I wonder if Arduino should get a API to do this instead…

Yes, you opened a can of worms, don’t be alarmed

[grazy]

Or you can sleep more than 7FFFFFFF ostick => 9h32min and even with rtc clock in hal_ticks you will have some problem (US_PER_OSTICK_EXPONENT = 4).

[EricVdB]

@arjanvanb

…or does it

LMiC’s TX_COMPLETE event takes 20-30 seconds to fire

the fact that micros() does not update might even be the cause of your problem in the first place: you queue a new packet after the duty cycle delay is over, but because you’ve been sleeping, LMIC doesn’t know that time has passed. Ideally, the micros() counter would be adapted to compensate for the sleep time which requires some changing some global variable internal to the Arduino code, but that might not be so portable. I wonder if Arduino should get a API to do this instead…

Yes, you opened a can of worms, don’t be alarmed

Again, that is only valid for AVR micro’s, not for the ESPx.

[Amedee]

@EricVdB

Again, that is only valid for AVR micro’s, not for the ESPx.

Right, this is a generic LMIC thread, not an ESP one…

[bluejedi]

Low-power and sleep are so essential that it should just work and be simple.

[Amedee]

Yes, and it seems that this combination is the best kept secret

[Verkehrsrot]

Is this solution still state-of-the-art, using it with MCCI LMIC? Or is there a better piece of code for the task of storing/restoring LMIC state while sleeping on ESP32?

/cc @grazy

[Milbrot]

In V3 OTAA is the preferred method to join LoRaWAN network.
It is now a good opportunity to switch my devices from ABP to OTAA.
That’s why I would like to know what is required to restore a session after power down the device.
In the end, was there a solution for MCCI LMIC and, for example, an ATmega328P?

[descartes (Nick McCloud)]

MCCI LMIC (and the older classic) show the keys returned when a join request is completed, those need preserving along with the frame count.

It is quite rare for anyone to power down their nodes - particularly on the lightweight AVRs which don’t consume much current at all in sleep mode - so I’m not aware of any implementations but it’s easy enough to code using the built-in flash.

You only need to store the keys each time you get a join accept event - but the frame counter will need storing before power down.

[kersing (Jac Kersing)]

@descartes

MCCI LMIC (and the older classic) show the keys returned when a join request is completed, those need preserving along with the frame count.

You need to store a bit more, channel definitions provided in the join accept come to mind, RX1 offset, ADR results (power and data rate), DevNonces used for join. There is probably more…
It would be easier to just use a incrementing DevNonce and only save that value and rejoin on reset. Be aware that limits the number of resets to just 65535 times before you need to re-add the device in TTN to generate a fresh set of ‘credentials’.

[Milbrot]

Thanks for the answers but it is hard to follow the best practices guidelines which says, for example:

“A device should keep the result of an activation in permanent storage
if the device is expected to be power-cycled during its lifetime.”

when the required information is not available or difficult to obtain to implement them.
For a “normal” user, this makes creating a device a trial and error process.
There should be a simple guide where it is described what to save, as always it doesn’t seem easy.

[bluejedi]

@Milbrot

There should be a simple guide where it is described what to save, as always it doesn’t seem easy.

I agree that good and complete documentation with clear steps and practical guidance for this is still missing.

If that would be available with clear working examples (if even in pseudo code) it would be much easier to create fully compliant devices.

Why isn’t something like that still not available after many years? Conlicting commercial interests?