Over-the-air-activation OTAA with LMIC

Awesome, I’ve just tested it, and patience is the key :smile:

Thanks a lot everyone!!

Yeah the node is 30 meters away from the gateway. With ABP it has always been a good connection, even at a distance of 1.5 km. Same goes for the RN2483 with OTAA, which instantly joins. Thus something different happens when I try the RFM95W with OTAA and this library. It could be that it fails to join and retries at a lower datarate, however this shouldn’t be necessary as the connection is good.

In the LoRaWAN 1.0 specification the duty cycle for joins in EU is indeed 0.1%. So if the initial joins fails for whatever reason, it will take a long time before it sends another attempt.

The LoRaWAN 1.0.1 update has relaxed this limitation to 1% join duty cycle in the first hour.

The reason for missing some join (accepts) is probably because the TTN backend can respond in the RX2 window and then uses data rate SF9. This is not compliant, and I just opened an issue for it. Before the join, the backend should respond with the LoRaWAN default values (=SF12 for RX2), and then it can change from the defaults by setting the new configuration parameters in the join accept message. After the join accept, it can then use the TTN values (=SF9 for RX2).

Also note that LMiC based end-node could miss a join accept because of incorrect RX slot timing, if your code does other stuff than just LMiC.

2 Likes

I guess you know, so just to be sure/for future readers: with ABP no join request is sent at all. Instead, all keys needed for the communication are already programmed into the node, hence no need for any join sequence.

1 Like

You are right, that was a poor choice of words. I changed it.

@Thomas

think you are (partially) right. When I changed the duty cycle form 0.1% to 1% the joining process was faster. However as expected, it still isn’t instantaneously. If the node retries to join at a lower data rate, is there anyway to see at which datarate it is trying or at which is successfully joined?

Furthemore changing RX2 window to SF9 or leaving it at SF12 (default) didn’t seem to make a difference.

LMIC.dn2Dr = SF9;

btw my code doesn’t do anything else then sending, it is pretty much the default example code.

I (partially) figured out why a node with the LMIC library, e.g. the RFM95W doesn’t join instantly when using OTAA. The standard code will first try to join the network at SF7 at certain frequency. However for some reason this will fail. After a failed attempt the node will try another frequency with the same datarate (SF7), this will fail again. Now the node will decrease the datarate to SF8 and again do two attempts which both will fail. Consequently the node decreases the datarate to SF9. This time the node will succeed at joining at the first attempt with this datarate.

After reading this comment of @kersing it seems explainable why SF7 and SF8 are not working (for me).

SF7 and SF8 should have a join reply in RX1 using the parameters of the request. SF9 + response is in RX2 at SF12

Based on this, reporting of others who also can’t join instantly with LMIC OTAA, and on tests with the Microchip RN2483, I conclude that the LMIC library doesn’t parse the received data in RX1 (correctly), please correct me if I am wrong, @matthijs @Thomas? As I will be on holidays for a while I can’t dig deeper into the code so I hope this information will help.

.
p.s. If you want a dirty fix for the instant join you can adjust the LMIC library to start with at SF9 by editing the following at line 686 in the LMIC library:

setDrJoin(DRCHG_SET, DR_SF7);

Change to:

setDrJoin(DRCHG_SET, DR_SF9);

1 Like

Keep in mind TTN does reply to join requests in RX2 at SF12, not SF9 as it previously did (that issue was fixed some time ago)

That is correct, that is also how my configuration is setup. However as I can’t join with SF7 and SF8, I suspect TTN does not send a reply join request in RX2 at SF12 but a reply in RX1 at SF7 and SF8 when using SF7 or SF8. Is this correct?

Correct, TTN sends join reply in RX1 if the gateway has available airtime and the request was SF7 or SF8.

I think I already joined a network on RX1@SF8 but I just tried and I am not able to reproduce it :confused:
Maybe a timing issue, I will search it too

I am not sure if I understand : it means that TTN only answers in RX1 for SF7 and SF8, and only RX2 for SF9 to SF12 ? If so, why this limitation ? Save airtime ?

1 Like

Thanks!

A life saver, but still pretty unstable, 50/50 it works, but when it does it always connects in max 20 seconds.

I just found a bug in the LMIC library, where it uses an invalid channel for sending join requests 50% of the time (because the list of join channels was reduced from 6 to the 3 in the LoRaWAN standard, but two places didn’t get updated properly). This means that in 50% of the cases, the first (and possibly second and third) join request would not be transmitted correctly. Once the channel wraps around from 5 to 0, it would be valid again and joining could again work. I’m committing a fix now, I expect to push it to my git repo today.

Note that this problem is not related to the SF used, so this does not explain what @Kibet is reporting (but perhaps this was the problem he was seeing and the channel selection just accidentally lined up with the SF selection). I haven’t been able to test joining at SF7 yet, since I’m too far away from my gateway for that to work now.

3 Likes

Also, there was another bug where, if joining was started automatically and the join request needed retransmission, the queued data packet would also be retransmitted a number of times (until an ack is received, but no ack was actually requested). This could cause the first EV_TXCOMPLETE after an EV_JOINED to be delayed for quite some time (up to 8 attempts with increasing SF’s). I’ll also have this fixed today.

3 Likes

Thanks for those bug corrections, did you pushed them already ?

Also I saw your comments on the license change, does it means that IBM will maintain LMIC or “just” allow a more permissive use of the stack ?
And I am not good in licenses, but does it also helps with the AES libraries you have tested ?

@Oliv, I didn’t get around to pushing them before, but I just did. I posted an update in another topic, I think that should answer your questions.

@Kibet, I just pushed quite some fixes to the lmic master branch, could you see if your join at SF7/SF8 persists? I’ve just tested this here, SF7 joins work for me. Two related remarks:

  • Perhaps your clock is inaccurate, but the error is negligable at the slower SF ratings? To check, add this to your sketch (somewhere during setup):

    LMIC_setClockError(MAX_CLOCK_ERROR * 1 / 100);

This tells LMIC to make the receive windows bigger, in case your clock is 1% faster or slower.

  • If you set LMIC.dn2Dr before starting a join, I believe the RX2 window during the join will use that datarate. I do not think that TTN will use RX2 for join accepts (since the RX1 should always have equal or lower SF, so less airtime), but perhaps I’m mistaken and it does matter somehow.
2 Likes

Tested with the latest merged code from matthijs.
Without the above patch (which i also did at line 879), the join process is faster, but still takes around 7 minutes.
I also noticed that the join request is now only performed on a few channels, not all.
Still, when the join process succeeds, it is always right after the request on 868.1Mhz
As experienced with a previous test
(LMIC-1.51 fitting in Arduino Atmega 328)

With the above patch from @Kibet , it is immediately (tested now 3x in a row)

EDIT:

@matthijs, your right!, setting the clock speed adter LMIC_reset() solved the problem also!
No need to patch the LMIC code

Ok, good to know. This would suggest either an inaccurate clock, or perhaps other (software) issues that are solved by increasing the receive window length. What hardware are you using? Is your sketch doing significant work between calls to osrunloop_once?

I would not expect this behaviour with git master, even when the clock is inaccurate (though perhaps it was just a coincidence).

If you want to dig in further, try setting LMIC_DEBUG_LEVEL to 2 (and uncomment the LMIC_PRINTF_TO line as well) to get more info about transmissions and settings used.[quote=“gizmocuz, post:37, topic:1921”]
I also noticed that the join request is now only performed on a few channels, not all.
[/quote]

The current version of LMIC only tries joining on the 868.1/3/5 Mhz channels. The LoRaWAN spec has three more frequencies, but the default gateway configurations and the TTN frequency plan does not include these frequencies, so LMIC does not use them.

My sketch is 99% your TTN-OTAA sketch, except it sends a text like “test 1”, “test 2”, etc…
It is not doing anything else at the moment

I now started the sketch about 20 times during the day, and all times the joining was instantly…

I am currently running some other tests, but tomorrow i will remove the clock fix, and will debug more with level 2, and report back. (But it’s working great now!, many thanks to you and all who made the pull requests)

1 Like

I did the test, this is the result without the clock adjustment call:

Starting OTAA New sample
RXMODE_RSSI
168: engineUpdate, opmode=0x808
Packet queued
1243: EV_JOINING
2412: engineUpdate, opmode=0xc
131206: engineUpdate, opmode=0xc
131511: TXMODE, freq=868100000, len=23, SF=7, BW=125, CR=4/5, IH=0
445987: RXMODE_SINGLE, freq=868100000, SF=7, BW=125, CR=4/5, IH=0
511463: RXMODE_SINGLE, freq=869525000, SF=12, BW=125, CR=4/5, IH=0
521599: engineUpdate, opmode=0xc
4187723: engineUpdate, opmode=0xc
4188029: TXMODE, freq=868300000, len=23, SF=7, BW=125, CR=4/5, IH=0
4502376: RXMODE_SINGLE, freq=868300000, SF=7, BW=125, CR=4/5, IH=0
4567853: RXMODE_SINGLE, freq=869525000, SF=12, BW=125, CR=4/5, IH=0
4577925: engineUpdate, opmode=0xc
8772865: engineUpdate, opmode=0xc
8773171: TXMODE, freq=868500000, len=23, SF=8, BW=125, CR=4/5, IH=0
9090827: RXMODE_SINGLE, freq=868500000, SF=8, BW=125, CR=4/5, IH=0
9156207: RXMODE_SINGLE, freq=869525000, SF=12, BW=125, CR=4/5, IH=0
9166280: engineUpdate, opmode=0xc
16773798: engineUpdate, opmode=0xc
16774106: TXMODE, freq=868100000, len=23, SF=8, BW=125, CR=4/5, IH=0
17091570: RXMODE_SINGLE, freq=868100000, SF=8, BW=125, CR=4/5, IH=0
17156950: RXMODE_SINGLE, freq=869525000, SF=12, BW=125, CR=4/5, IH=0
17166957: engineUpdate, opmode=0xc
24745192: engineUpdate, opmode=0xc
24745500: TXMODE, freq=868300000, len=23, SF=9, BW=125, CR=4/5, IH=0
25068940: RXMODE_SINGLE, freq=868300000, SF=9, BW=125, CR=4/5, IH=0
25134128: RXMODE_SINGLE, freq=869525000, SF=12, BW=125, CR=4/5, IH=0
25245291: Setup channel, idx=3, freq=867100000
25245336: Setup channel, idx=4, freq=867300000
25247185: Setup channel, idx=5, freq=867500000
25250240: Setup channel, idx=6, freq=867700000
25253295: Setup channel, idx=7, freq=867900000
25256502: EV_JOINED
netid = 921102
25258759: engineUpdate, opmode=0x808
25261524: TXMODE, freq=867100000, len=20, SF=9, BW=125, CR=4/5, IH=0
Packet queued
25262523: engineUpdate, opmode=0x888
25331831: RXMODE_SINGLE, freq=867100000, SF=9, BW=125, CR=4/5, IH=0
25394330: RXMODE_SINGLE, freq=869525000, SF=9, BW=125, CR=4/5, IH=0
25643508: EV_TXCOMPLETE (includes waiting for RX windows)
25643556: engineUpdate, opmode=0x800

This is the test with the clock adjustment call:

Starting OTAA New sample
RXMODE_RSSI
224: engineUpdate, opmode=0x808
Packet queued
1298: EV_JOINING
2467: engineUpdate, opmode=0xc
199891: engineUpdate, opmode=0xc
200195: TXMODE, freq=868500000, len=23, SF=7, BW=125, CR=4/5, IH=0
511503: RXMODE_SINGLE, freq=868500000, SF=7, BW=125, CR=4/5, IH=0
519002: Setup channel, idx=3, freq=867100000
519045: Setup channel, idx=4, freq=867300000
520635: Setup channel, idx=5, freq=867500000
523560: Setup channel, idx=6, freq=867700000
526485: Setup channel, idx=7, freq=867900000
529563: EV_JOINED
netid = 921102
531689: engineUpdate, opmode=0x808
534322: TXMODE, freq=867100000, len=20, SF=7, BW=125, CR=4/5, IH=0
Packet queued
535321: engineUpdate, opmode=0x888
596272: RXMODE_SINGLE, freq=867100000, SF=7, BW=125, CR=4/5, IH=0
658516: RXMODE_SINGLE, freq=869525000, SF=9, BW=125, CR=4/5, IH=0
951196: EV_TXCOMPLETE (includes waiting for RX windows)
951239: engineUpdate, opmode=0x800
1 Like