How to persist LMIC OTAA parameters with an ESP32?

Hi everyone,

I’m searching for a simple method to persist the message counters and other values from an OTAA join. Does anybody know a good LoraWAN library which persists all required values or how to persist and load the required values from LMiC? My goal is to use OTAA in combination with an ESP32 and deep sleep.

First I thought this is a common question but unfortunately I didn’t find a solution…

Thank you very much :slight_smile:

@JackGruber Could show in code where / when you call this

Nice. You may want to check if you can save the state for confirmed downlinks. And if you do, then you’re doing better than the RN2483. :slight_smile:

Load the settings in the Lora Setup

    ...
    LMIC_reset();

    if(RTC_LORAWAN_seqnoUp != 0)
    {
        LoraWANLoadOTTAFromRTC();
    }

    // Start job
    LoraWANDo_send(&sendjob); 

and after EV_TXCOMPLETE i send my ESP to deep sleep.

At the moment i have no access to my devices. But in i 4 weeks i think i take a look into this.

1 Like

Thanks for this post.
About 4 weeks ago I was in the same situation and was asking myself why there is no proper Demo Sketch of video for that.
I was very frustated during my programing to do on every small change a new OTAA Activation.
I read a lot of forum discussions with a lot of tips and tricks and finally I found something useful.

After a lot of searching i found this sketch for the EPS8266: https://github.com/Edzelf/LoRa/tree/master/ESP_lora_tracker

Once inspired I modified this sketch according to my needs to ESP32:
I modified the EEPROM Access and the RTC handling.
You may find my modification here:

I am not an expert programmer and hope this sketch helps you. In my opinion there should be much more of this basic examples documented.
Hopefully somebody make this code more cleaner and add this to MCCI stack examples.

1 Like

LDL passes devNonce and session state back to the application via a callback. The application can choose to save and/or restore this as needed. There is more information in the porting guide.

Unfortunately there is no LDL wrapper for ESP32 Arduino at this time, so it’s not super convenient to try out. Even if there was a wrapper it probably wouldn’t persist by default since it makes experimenting with LoRaWAN more difficult.

Hi @arjanvanb,

i have checked my code on a short test and the ack for the downlink works fine after deep sleep.
image
I think the downlink retrys came from a timing problem, because it was the same with or without deepsleep on my ESP32.

1 Like

Some remarks:

  • Be aware that ESP32 has no EEPROM and the data will be stored in flash.
    Flash has a limited lifetime of around 100k erase/program cycles. It should therefore not be used to store data that is very frequently changing.

  • The EEPROM library for ESP32 is deprecated.

    For new applications on ESP32, use Preferences.

  • About the preferences library:

    Preferences provides persistent (across resets) but mutable storage of various types of variables. It is similar to EEPROM library in Arduino, except that EEPROM provides a single contiguous block of storage which the sketch needs to partition between variables, while Preferences performs the partitioning itself.

    (I have not used the preferences library yet.)

1 Like

EDIT: DO NOT USE THIS CODE. ITS WRONG!
Description in the next post below

Hey together,
sorry for my really late response. I was very busy the last days.

Thank you very much @JackGruber! This helped me a lot!
Everything worked for me except the handling of confirmed downlinks.

I found out that we need to persist the LMIC.dnConf too.

The parameter is declared in the lmic.h file with a comment:

u1_t        dnConf;       // dn frame confirm pending: LORA::FCT_ACK or 0

When loading the data back from the RTC ists important to set the dnConf attribute after calling the LMIC_set... methods because they override this attribute.

I expanded the code from @JackGruber with three lines.

// EDIT: DO NOT USE THIS CODE. ITS WRONG!
// Description in the next post below
//
RTC_DATA_ATTR u4_t RTC_LORAWAN_netid = 0;
RTC_DATA_ATTR devaddr_t RTC_LORAWAN_devaddr = 0;
RTC_DATA_ATTR u1_t RTC_LORAWAN_nwkKey[16];
RTC_DATA_ATTR u1_t RTC_LORAWAN_artKey[16];
RTC_DATA_ATTR u4_t RTC_LORAWAN_seqnoUp = 0;
RTC_DATA_ATTR u4_t RTC_LORAWAN_seqnoDn;
RTC_DATA_ATTR u1_t RTC_LORAWAN_dn2Dr;
RTC_DATA_ATTR u1_t RTC_LORAWAN_dnConf;
RTC_DATA_ATTR s1_t RTC_LORAWAN_adrTxPow;
RTC_DATA_ATTR s1_t RTC_LORAWAN_datarate;
RTC_DATA_ATTR u4_t RTC_LORAWAN_channelFreq[MAX_CHANNELS];
RTC_DATA_ATTR u2_t RTC_LORAWAN_channelDrMap[MAX_CHANNELS];
RTC_DATA_ATTR u2_t RTC_LORAWAN_channelMap;
RTC_DATA_ATTR s2_t RTC_LORAWAN_adrAckReq;
RTC_DATA_ATTR u1_t RTC_LORAWAN_rx1DrOffset;
RTC_DATA_ATTR u1_t RTC_LORAWAN_rxDelay;

void LoraWANSaveOTTA2RTC()
{
    Serial.println(F("Save LMIC to RTC ..."));
    RTC_LORAWAN_netid = LMIC.netid;
    RTC_LORAWAN_devaddr = LMIC.devaddr;
    memcpy(RTC_LORAWAN_nwkKey, LMIC.nwkKey, 16);
    memcpy(RTC_LORAWAN_artKey, LMIC.artKey, 16);
    RTC_LORAWAN_dn2Dr = LMIC.dn2Dr;
    RTC_LORAWAN_dnConf = LMIC.dnConf;
    Serial.println(RTC_LORAWAN_dnConf);
    RTC_LORAWAN_seqnoDn = LMIC.seqnoDn;
    RTC_LORAWAN_seqnoUp = LMIC.seqnoUp;
    RTC_LORAWAN_adrTxPow = LMIC.adrTxPow;
    RTC_LORAWAN_datarate = LMIC.datarate;
    RTC_LORAWAN_adrAckReq = LMIC.adrAckReq;
    RTC_LORAWAN_rx1DrOffset = LMIC.rx1DrOffset;
    RTC_LORAWAN_rxDelay = LMIC.rxDelay;
    memcpy(LMIC.channelFreq, RTC_LORAWAN_channelFreq, MAX_CHANNELS);
    memcpy(LMIC.channelDrMap, RTC_LORAWAN_channelDrMap, MAX_CHANNELS);
    RTC_LORAWAN_channelMap = LMIC.channelMap;
}

void LoraWANLoadOTTAFromRTC()
{
    Serial.println(F("Load LMIC from RTC ..."));

    memcpy(RTC_LORAWAN_channelFreq, LMIC.channelFreq, MAX_CHANNELS);
    memcpy(RTC_LORAWAN_channelDrMap, LMIC.channelDrMap, MAX_CHANNELS);
    LMIC.channelMap = RTC_LORAWAN_channelMap;
    LMIC.seqnoDn = RTC_LORAWAN_seqnoDn;
    LMIC_setSession(RTC_LORAWAN_netid, RTC_LORAWAN_devaddr, RTC_LORAWAN_nwkKey, RTC_LORAWAN_artKey);
    LMIC_setSeqnoUp(RTC_LORAWAN_seqnoUp);
    LMIC_setDrTxpow(RTC_LORAWAN_datarate, RTC_LORAWAN_adrTxPow);
    LMIC.dnConf = RTC_LORAWAN_dnConf;
    LMIC.adrAckReq = RTC_LORAWAN_adrAckReq;
    LMIC.dn2Dr = RTC_LORAWAN_dn2Dr;
    LMIC.rx1DrOffset = RTC_LORAWAN_rx1DrOffset;
    LMIC.rxDelay = RTC_LORAWAN_rxDelay;
}

@arjanvanb Did you mean this with “save the state for confirmed downlinks”?

2 Likes

@JackGruber It seems like that it was only luck that your code did something. I wondered that my node only used one channel (with your code modified in the post above this one ) and started investigating

First of all I noticed that the array with the bands is missing which stores the duty cycles and the usages of the channel etc.

When I was typing the corresponding memcpy command I noticed that you switched the destination and the source parameter in your code . Also MAX_CHANNELS as the size parameter is wrong because the command wants a size in bytes and not the count of array entries. So we need to multiply MAX_CHANNELS with the size of the array entries:

memcpy(RTC_LORAWAN_channelDrMap, LMIC.channelDrMap, MAX_CHANNELS*sizeof(u2_t));

It would be nice if you change this in your code block so that other readers of this thread don’t copy a switched memcpy :slight_smile:

Currently this code block is working ( I did something with the duty cycle because after deep sleep the ESP resets the system clock):

RTC_DATA_ATTR u4_t RTC_LORAWAN_netid = 0;
RTC_DATA_ATTR devaddr_t RTC_LORAWAN_devaddr = 0;
RTC_DATA_ATTR u1_t RTC_LORAWAN_nwkKey[16];
RTC_DATA_ATTR u1_t RTC_LORAWAN_artKey[16];
RTC_DATA_ATTR u4_t RTC_LORAWAN_seqnoUp = 0;
RTC_DATA_ATTR u4_t RTC_LORAWAN_seqnoDn;
RTC_DATA_ATTR u1_t RTC_LORAWAN_dn2Dr;
RTC_DATA_ATTR u1_t RTC_LORAWAN_dnConf;
RTC_DATA_ATTR s1_t RTC_LORAWAN_adrTxPow;
RTC_DATA_ATTR u1_t RTC_LORAWAN_txChnl;
RTC_DATA_ATTR s1_t RTC_LORAWAN_datarate;
RTC_DATA_ATTR u4_t RTC_LORAWAN_channelFreq[MAX_CHANNELS];
RTC_DATA_ATTR u2_t RTC_LORAWAN_channelDrMap[MAX_CHANNELS];
RTC_DATA_ATTR u4_t RTC_LORAWAN_channelDlFreq[MAX_CHANNELS];
RTC_DATA_ATTR band_t RTC_LORAWAN_bands[MAX_BANDS];
RTC_DATA_ATTR u2_t RTC_LORAWAN_channelMap;
RTC_DATA_ATTR s2_t RTC_LORAWAN_adrAckReq;
RTC_DATA_ATTR u1_t RTC_LORAWAN_rx1DrOffset;
RTC_DATA_ATTR u1_t RTC_LORAWAN_rxDelay;



void Sleep(int sleepSeconds)
{
    Serial.println(F("Save LMIC to RTC ..."));
    RTC_LORAWAN_netid = LMIC.netid;
    RTC_LORAWAN_devaddr = LMIC.devaddr;
    memcpy(RTC_LORAWAN_nwkKey, LMIC.nwkKey, 16);
    memcpy(RTC_LORAWAN_artKey, LMIC.artKey, 16);
    RTC_LORAWAN_dn2Dr = LMIC.dn2Dr;
    RTC_LORAWAN_dnConf = LMIC.dnConf;
    RTC_LORAWAN_seqnoDn = LMIC.seqnoDn;
    RTC_LORAWAN_seqnoUp = LMIC.seqnoUp;
    RTC_LORAWAN_adrTxPow = LMIC.adrTxPow;
    RTC_LORAWAN_txChnl = LMIC.txChnl;
    RTC_LORAWAN_datarate = LMIC.datarate;
    RTC_LORAWAN_adrAckReq = LMIC.adrAckReq;
    RTC_LORAWAN_rx1DrOffset = LMIC.rx1DrOffset;
    RTC_LORAWAN_rxDelay = LMIC.rxDelay;
    memcpy(RTC_LORAWAN_channelFreq, LMIC.channelFreq, MAX_CHANNELS*sizeof(u4_t));
    memcpy(RTC_LORAWAN_channelDrMap, LMIC.channelDrMap, MAX_CHANNELS*sizeof(u2_t));
    memcpy(RTC_LORAWAN_channelDlFreq, LMIC.channelDlFreq, MAX_CHANNELS*sizeof(u4_t));
    memcpy(RTC_LORAWAN_bands, LMIC.bands, MAX_BANDS*sizeof(band_t));
    RTC_LORAWAN_channelMap = LMIC.channelMap;
    
    //System time is resetted after sleep. So we need to calculate the dutycycle with a resetted system time
    delay(random(1000));
    unsigned long now = millis();
    Serial.println(now);
    for(int i = 0; i < MAX_BANDS; i++) {
        ostime_t correctedAvail = RTC_LORAWAN_bands[i].avail - ((now/1000.0 + sleepSeconds ) * OSTICKS_PER_SEC);
        if(correctedAvail < 0) {
            correctedAvail = 0;
        }
        RTC_LORAWAN_bands[i].avail = correctedAvail;
    }
    esp_sleep_enable_timer_wakeup(1000000 * sleepSeconds);
    esp_deep_sleep_start();
}

void Wakeup()
{
    Serial.println(F("Load LMIC from RTC ..."));
    
    LMIC_setSession(RTC_LORAWAN_netid, RTC_LORAWAN_devaddr, RTC_LORAWAN_nwkKey, RTC_LORAWAN_artKey);
    LMIC_setSeqnoUp(RTC_LORAWAN_seqnoUp);
    LMIC_setDrTxpow(RTC_LORAWAN_datarate, RTC_LORAWAN_adrTxPow);
    memcpy(LMIC.bands, RTC_LORAWAN_bands, MAX_BANDS*sizeof(band_t));
    memcpy(LMIC.channelFreq, RTC_LORAWAN_channelFreq, MAX_CHANNELS*sizeof(u4_t));
    memcpy(LMIC.channelDlFreq, RTC_LORAWAN_channelDlFreq, MAX_CHANNELS*sizeof(u4_t));
    memcpy(LMIC.channelDrMap, RTC_LORAWAN_channelDrMap, MAX_CHANNELS*sizeof(u2_t));
    LMIC.seqnoDn = RTC_LORAWAN_seqnoDn;
    LMIC.dnConf = RTC_LORAWAN_dnConf;
    LMIC.adrAckReq = RTC_LORAWAN_adrAckReq;
    LMIC.dn2Dr = RTC_LORAWAN_dn2Dr;
    LMIC.rx1DrOffset = RTC_LORAWAN_rx1DrOffset;
    LMIC.rxDelay = RTC_LORAWAN_rxDelay;
    LMIC.txChnl = RTC_LORAWAN_txChnl;
    LMIC.channelMap = RTC_LORAWAN_channelMap;
}

Im sure that there are some attributes missing. I will add them in this post when I know about them :slight_smile:

Anyway: Thank you @JackGruber for giving me the right hint to persist the attributes in the RTC :slight_smile: This was really helpful!

1 Like

I think the following handles that?

However,

I wonder why that worked then, without that line of code. (And also: why the erroneous memcpy worked.) Or maybe the settings are simply not lost at all, in the sleep that is used for testing?

Oh my, the memcpy mistake is so embarrassing :see_no_evil:

yes, I already noticed that with the dutycycle.
In my ATmega projects I adjust the clock after the deepsleep.

// LMIC uses micros() to keep track of the duty cycle, so
// hack timer0_overflow for a rude adjustment: 
cli();
timer0_overflow_count+= 8 * 64 * clockCyclesPerMicrosecond();
sei();

I have not yet checked how this works with ESP32 and what variables we need to store.

I have deleted the old post, because editing was not possible.

2 Likes