# Multitech - TTN connection lost after bad connectivity

**URL:** <https://www.thethingsnetwork.org/forum/t/multitech-ttn-connection-lost-after-bad-connectivity/12355>\
**Category:** MultiTech Gateways\
**Created:** [February 5, 2018, 12:19pm UTC](https://www.thethingsnetwork.org/forum/t/multitech-ttn-connection-lost-after-bad-connectivity/12355 "2018-02-05T12:19:23Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Paul\_Stewart](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/paul_stewart/32/46922_2.png) [@Paul\_Stewart](https://www.thethingsnetwork.org/forum/u/Paul_Stewart)\
**Post date:** [February 5, 2018, 12:19pm UTC](https://www.thethingsnetwork.org/forum/t/multitech-ttn-connection-lost-after-bad-connectivity/12355/1 "2018-02-05T12:19:23Z")

</div>

Hi All,

I have a few Mulittech Conduits deployed that use internet connections that arent exactly reliable (on remote farms using 3g and 4g connections). All have @kersing 's “new” packet forwarder installed following this guide [https://www.thethingsnetwork.org/docs/gateways/multitech/aep.html](https://www.thethingsnetwork.org/docs/gateways/multitech/aep.html)

Sometimes the internet connection drops out for about an hour or more, and when the connection comes back up, although I can see the gateways in the Multitech [deviceshq.com](http://deviceshq.com) console to show they have an internet connection again, they aren’t seen in the TTN console again.

It normally takes a reboot of the gateway to get it connected again or a complete power cycle.

Could anybody explain what might be happening?

@kersing does it retry several times at different intervals and then give up and won’t try again?

any help appreciated

cheers  
Paul

---

<div class="post-metadata">

**Author:** ![pe1mew](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/pe1mew/32/45049_2.png) [@pe1mew](https://www.thethingsnetwork.org/forum/u/pe1mew)\
**Post date:** [February 5, 2018, 12:41pm UTC](https://www.thethingsnetwork.org/forum/t/multitech-ttn-connection-lost-after-bad-connectivity/12355/2 "2018-02-05T12:41:01Z")

</div>

I have a similar problem here.

---

<div class="post-metadata">

**Author:** ![kersing](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/kersing/32/44777_2.png) [@kersing](https://www.thethingsnetwork.org/forum/u/kersing)\
**Post date:** [February 5, 2018, 4:59pm UTC](https://www.thethingsnetwork.org/forum/t/multitech-ttn-connection-lost-after-bad-connectivity/12355/3 "2018-02-05T16:59:33Z")

</div>

> [@Paul\_Stewart](#):
>
> does it retry several times at different intervals and then give up and won’t try again?

No, it continuously retries. At least it should. However there is an obscure issue I’ve been trying to solve for 6 months now that results in this behavior. As I can’t reproduce it myself I’m working with community members to get to the bottom of it. (I’m pushing new sources with additional debugging while typing this)

---

<div class="post-metadata">

**Author:** ![pferland](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/pferland/32/35697_2.png) [@pferland](https://www.thethingsnetwork.org/forum/u/pferland)\
**Post date:** [February 5, 2018, 6:43pm UTC](https://www.thethingsnetwork.org/forum/t/multitech-ttn-connection-lost-after-bad-connectivity/12355/4 "2018-02-05T18:43:19Z")

</div>

A similar rare, intermittent issue occurs with the “Semtech UDP” packet forwarder built into the Conduit. A work around in that packet forwarder is setting an “autoquit\_threshold” in global\_conf.json under gateway\_conf. For example:  
{  
[…]  
“gateway\_conf”: {  
“autoquit\_threshold”: 5,  
“server\_address”: “router.eu.thethings.network”,  
“serv\_port\_up”: 1700,  
“serv\_port\_down”: 1700,  
[…]  
}  
}

After 5 failures the packet forwarder will exit and the watchdog included in the AEP OS will start another instance of the packet forwarder a few seconds later.

If the connectivity problem is only related to the PPP connection, you could add a line restarting the packet forwarder in /etc/ppp/ip-up. For the built in packet forwarder this could be done by adding the line “/etc/init.d/lora-packet-forwarder restart” to the bottom of the file.

---

<div class="post-metadata">

**Author:** ![pe1mew](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/pe1mew/32/45049_2.png) [@pe1mew](https://www.thethingsnetwork.org/forum/u/pe1mew)\
**Post date:** [February 6, 2018, 6:02am UTC](https://www.thethingsnetwork.org/forum/t/multitech-ttn-connection-lost-after-bad-connectivity/12355/5 "2018-02-06T06:02:17Z")

</div>

I would love to have such feature because it keeps me from watching the gateway daily and resetting it manually. However I prefer to know the cause.

It seems as if the software is suffering from the same bug?

---

<div class="post-metadata">

**Author:** ![ecoadapt](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/ecoadapt/32/35697_2.png) [@ecoadapt](https://www.thethingsnetwork.org/forum/u/ecoadapt)\
**Post date:** [February 6, 2018, 8:27am UTC](https://www.thethingsnetwork.org/forum/t/multitech-ttn-connection-lost-after-bad-connectivity/12355/6 "2018-02-06T08:27:35Z")

</div>

Hi guys,  
I have experienced the same issue several times with all my 3 TTN Multitech Conduit.  
Regards,  
Sylvain

---

<div class="post-metadata">

**Author:** ![cultsdotelecomatgmai](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/cultsdotelecomatgmai/32/35697_2.png) [@cultsdotelecomatgmai](https://www.thethingsnetwork.org/forum/u/cultsdotelecomatgmai)\
**Post date:** [February 6, 2018, 8:44am UTC](https://www.thethingsnetwork.org/forum/t/multitech-ttn-connection-lost-after-bad-connectivity/12355/7 "2018-02-06T08:44:23Z")

</div>

Hi @Paul_Stewart, I run a variety of unattended systems on RPi/Linux including a number of TTN gateways and a number of edge-computing “things”.  
I maintain availability using systemd to restart software and watchdog to restart the OS.  
I have not used the packet forwarder from @kersing but if the software exits with a non-zero status on encountering an abnormal situation, uses a PID file and has an option to regularly touch a file in /var/run when operating normally then it’s very easy to detect hangs and exits and automate restarts.

---

<div class="post-metadata">

**Author:** ![jreiss](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/jreiss/32/46529_2.png) [@jreiss](https://www.thethingsnetwork.org/forum/u/jreiss)\
**Post date:** [February 6, 2018, 3:14pm UTC](https://www.thethingsnetwork.org/forum/t/multitech-ttn-connection-lost-after-bad-connectivity/12355/8 "2018-02-06T15:14:42Z")

</div>

When the ppp connection is restored the IP address may have changed.  
This breaks the UDP socket connection between the gateway and network server.

The packet forwarder has a keepalive\_interval to send packets to the network server to allow downlinks to be sent at anytime. But this keepalive is only half configured without an autoquit\_threshold to bring down the socket or exit the process.

Using the autoquit\_threshold setting, as Peter mentined, will allow the process to exit when the network server is unavailable. When the process is restarted a new socket will be created on the changed interface.

This is also available in Kersing packet forwarder as it is based on Semtech code.

This can be tested over Ethernet by changing the ip address of the gateway.

Without the autoquit\_threshold the packet forwarder will keep receiving packets and send them over the broken socket.

When autoquit\_threshold is specified the process will quit and the socket reset to restore the connection automatically if there is an angel or monitor process to restart the packet forwarder.

---

<div class="post-metadata">

**Author:** ![Paul\_Stewart](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/paul_stewart/32/46922_2.png) [@Paul\_Stewart](https://www.thethingsnetwork.org/forum/u/Paul_Stewart)\
**Post date:** [February 7, 2018, 5:35am UTC](https://www.thethingsnetwork.org/forum/t/multitech-ttn-connection-lost-after-bad-connectivity/12355/9 "2018-02-07T05:35:05Z")

</div>

> [@pferland](#):
>
> A similar rare, intermittent issue occurs with the “Semtech UDP” packet forwarder built into the Conduit. A work around in that packet forwarder is setting an “autoquit\_threshold” in global\_conf.json under gateway\_conf. For example:
> 
> {
> 
> […]
> 
> “gateway\_conf”: {
> 
> “autoquit\_threshold”: 5,
> 
> “server\_address”: “router.eu.thethings.network”,
> 
> “serv\_port\_up”: 1700,
> 
> “serv\_port\_down”: 1700,
> 
> […]
> 
> }
> 
> }
> 
> After 5 failures the packet forwarder will exit and the watchdog included in the AEP OS will start another instance of the packet forwarder a few seconds later.

@kersing do you think this would work with your forwarder? (as a work around)

I have one gateway in particular in which this is a pretty regular problem, and if I could simply do this for now, that would be great

regards  
Paul

---

<div class="post-metadata">

**Author:** ![Paul\_Stewart](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/paul_stewart/32/46922_2.png) [@Paul\_Stewart](https://www.thethingsnetwork.org/forum/u/Paul_Stewart)\
**Post date:** [February 7, 2018, 6:04am UTC](https://www.thethingsnetwork.org/forum/t/multitech-ttn-connection-lost-after-bad-connectivity/12355/10 "2018-02-07T06:04:45Z")

</div>

> [@cultsdotelecomatgmai](#):
>
> Hi @Paul_Stewart, I run a variety of unattended systems on RPi/Linux including a number of TTN gateways and a number of edge-computing “things”.
> 
> I maintain availability using systemd to restart software and watchdog to restart the OS.
> 
> I have not used the packet forwarder from @kersing but if the software exits with a non-zero status on encountering an abnormal situation, uses a PID file and has an option to regularly touch a file in /var/run when operating normally then it’s very easy to detect hangs and exits and automate restarts.

@cultsdotelecomatgmai sounds like a good solution to the problem but it would be good to diagnose the original issue.

With my level of expertise (which I am already in over my head) I’ll see if the autoquit line will work first.

cheers  
Paul

---

<div class="post-metadata">

**Author:** ![pe1mew](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/pe1mew/32/45049_2.png) [@pe1mew](https://www.thethingsnetwork.org/forum/u/pe1mew)\
**Post date:** [February 7, 2018, 7:20am UTC](https://www.thethingsnetwork.org/forum/t/multitech-ttn-connection-lost-after-bad-connectivity/12355/11 "2018-02-07T07:20:47Z")

</div>

Same here. in a couple of weeks I will contact @kersing to see if I can contribute to find the root cause.

---

<div class="post-metadata">

**Author:** ![kersing](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/kersing/32/44777_2.png) [@kersing](https://www.thethingsnetwork.org/forum/u/kersing)\
**Post date:** [February 7, 2018, 11:51am UTC](https://www.thethingsnetwork.org/forum/t/multitech-ttn-connection-lost-after-bad-connectivity/12355/12 "2018-02-07T11:51:17Z")

</div>

I’m working on solving the issue. Temporarily on hold because I’ve got influenza. Reading a few mails can be done on a tablet in bed, coding not…

---

<div class="post-metadata">

**Author:** ![pe1mew](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/pe1mew/32/45049_2.png) [@pe1mew](https://www.thethingsnetwork.org/forum/u/pe1mew)\
**Post date:** [February 7, 2018, 11:55am UTC](https://www.thethingsnetwork.org/forum/t/multitech-ttn-connection-lost-after-bad-connectivity/12355/13 "2018-02-07T11:55:42Z")

</div>

Beterschap!, Doe rustig aan, er is geen haast bij. Het is maar hobby.

---

<div class="post-metadata">

**Author:** ![Paul\_Stewart](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/paul_stewart/32/46922_2.png) [@Paul\_Stewart](https://www.thethingsnetwork.org/forum/u/Paul_Stewart)\
**Post date:** [February 7, 2018, 12:34pm UTC](https://www.thethingsnetwork.org/forum/t/multitech-ttn-connection-lost-after-bad-connectivity/12355/14 "2018-02-07T12:34:34Z")

</div>

Get better soon @kersing

We all really appreciate the work you do, I don’t think I would have 7 gateways operating on TTN if it wasn’t for you.

Cheers Paul

---

<div class="post-metadata">

**Author:** ![ecoadapt](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/ecoadapt/32/35697_2.png) [@ecoadapt](https://www.thethingsnetwork.org/forum/u/ecoadapt)\
**Post date:** [February 19, 2018, 2:23pm UTC](https://www.thethingsnetwork.org/forum/t/multitech-ttn-connection-lost-after-bad-connectivity/12355/15 "2018-02-19T14:23:52Z")

</div>

Hi all,

I still have disconnections of my Multitech AEP gateways from the TTN server. I have to reboot the gateway each time it occurs. Don’t know what is the root cause, probably not even a connectivity issue.  
Below are the last packet forwarder logs. Disconnection occured at this time or a bit before. No logs after this, until I rebooted 3 hours later.  
Does anyone has an idea of what could be the issue and/or how to recover automaticaly from it ?

Regards,  
Sylvain

admin@mtcdt:~# tail -f /var/log/lora-pkt-fwd.log  
12:09:42 INFO: Flush output after statistic is disabled  
12:09:42 INFO: Flush after each line of output is disabled  
12:09:42 INFO: Watchdog is disabled  
12:09:42 INFO: Contact email configured to ""  
12:09:42 INFO: Description configured to ""  
12:09:42 INFO: [Transports] Initializing protocol for 1 servers  
12:09:58 ERROR: [TTN] Connection to server “bridge.eu.thethings.network” failed, retry in 30 seconds  
12:11:02 ERROR: [TTN] Connection to server “bridge.eu.thethings.network” failed, retry in 60 seconds  
12:11:50 ERROR: [TTN] Connection to server “bridge.eu.thethings.network” failed, retry in 30 seconds  
12:12:33 INFO: [TTN] server “bridge.eu.thethings.network” connected  
^C  
admin@mtcdt:~# date  
Mon Feb 19 14:54:14 CET 2018

---

<div class="post-metadata">

**Author:** ![ecoadapt](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/ecoadapt/32/35697_2.png) [@ecoadapt](https://www.thethingsnetwork.org/forum/u/ecoadapt)\
**Post date:** [February 19, 2018, 3:10pm UTC](https://www.thethingsnetwork.org/forum/t/multitech-ttn-connection-lost-after-bad-connectivity/12355/16 "2018-02-19T15:10:39Z")

</div>

Actually, same symptoms today roughly at the same time, with my 3 TTN gateways. Located at different places, so not an issue with the local connectivity, but rather related to a root cause on the TTn server side. How to gather information on server faults ? Anyway whaterver the server issue, the gateways should have recovered from this…

---

<div class="post-metadata">

**Author:** ![kersing](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/kersing/32/44777_2.png) [@kersing](https://www.thethingsnetwork.org/forum/u/kersing)\
**Post date:** [February 19, 2018, 6:08pm UTC](https://www.thethingsnetwork.org/forum/t/multitech-ttn-connection-lost-after-bad-connectivity/12355/17 "2018-02-19T18:08:18Z")

</div>

Please be patient for a couple more days, I running (hopefully) final tests on an update that _should_ solve these issues.

---

<div class="post-metadata">

**Author:** ![Paul\_Stewart](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/paul_stewart/32/46922_2.png) [@Paul\_Stewart](https://www.thethingsnetwork.org/forum/u/Paul_Stewart)\
**Post date:** [February 19, 2018, 10:07pm UTC](https://www.thethingsnetwork.org/forum/t/multitech-ttn-connection-lost-after-bad-connectivity/12355/18 "2018-02-19T22:07:19Z")

</div>

great work !

---

<div class="post-metadata">

**Author:** ![ecoadapt](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/ecoadapt/32/35697_2.png) [@ecoadapt](https://www.thethingsnetwork.org/forum/u/ecoadapt)\
**Post date:** [February 20, 2018, 8:28am UTC](https://www.thethingsnetwork.org/forum/t/multitech-ttn-connection-lost-after-bad-connectivity/12355/19 "2018-02-20T08:28:14Z")

</div>

Thanks, I look forward to testing it !

---

<div class="post-metadata">

**Author:** ![ecoadapt](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/ecoadapt/32/35697_2.png) [@ecoadapt](https://www.thethingsnetwork.org/forum/u/ecoadapt)\
**Post date:** [February 21, 2018, 10:52am UTC](https://www.thethingsnetwork.org/forum/t/multitech-ttn-connection-lost-after-bad-connectivity/12355/20 "2018-02-21T10:52:21Z")

</div>

Hi,

Once again connectivity of one my TTN gateways was lost 18 hours ago. But this time -unlike the last issue- I still have logs going (see below). No message regarding connection to TTN though.  
Do you think it is another symptom of the same problem?

```
##### 2018-02-21 10:43:45 GMT #####
### [UPSTREAM] ###
# RF packets received by concentrator: 0
# CRC_OK: 0.00%, CRC_FAIL: 0.00%, NO_CRC: 0.00%
# RF packets forwarded: 0 (0 bytes)
# PUSH_DATA datagrams sent: 0 (0 bytes)
# PUSH_DATA acknowledged: 0.00%
### [DOWNSTREAM] ###
# PULL_DATA sent: 0 (0.00% acknowledged)
# PULL_RESP(onse) datagrams received: 0 (0 bytes)
# RF packets sent to concentrator: 0 (0 bytes)
# TX errors: 0
### BEACON IS DISABLED!
### [JIT] ###
# INFO: JIT queue contains 0 packets.
# INFO: JIT queue contains 0 beacons.
### GPS IS DISABLED!
### [PERFORMANCE] ###
# Upstream radio packet quality: 0.00%.
# Semtech status report send.
##### END #####

##### 2018-02-21 10:44:16 GMT #####
### [UPSTREAM] ###
# RF packets received by concentrator: 5
# CRC_OK: 100.00%, CRC_FAIL: 0.00%, NO_CRC: 0.00%
# RF packets forwarded: 5 (108 bytes)
# PUSH_DATA datagrams sent: 0 (0 bytes)
# PUSH_DATA acknowledged: 0.00%
### [DOWNSTREAM] ###
# PULL_DATA sent: 0 (0.00% acknowledged)
# PULL_RESP(onse) datagrams received: 0 (0 bytes)
# RF packets sent to concentrator: 0 (0 bytes)
# TX errors: 0
### BEACON IS DISABLED!
### [JIT] ###
# INFO: JIT queue contains 0 packets.
# INFO: JIT queue contains 0 beacons.
### GPS IS DISABLED!
### [PERFORMANCE] ###
# Upstream radio packet quality: 100.00%.
# Semtech status report send.
##### END #####
11:44:42 INFO: Disabling GPS mode for concentrator's counter...
11:44:42 INFO: host/sx1301 time offset=(1519207867s:282111µs) - drift=-805µs
11:44:42 INFO: Enabling GPS mode for concentrator's counter.

lgw_receive:1165: FIFO content: 1 99 0 5 22
lgw_receive:1184: [2 17]
Note: LoRa packet
lgw_receive:1165: FIFO content: 1 cb 0 5 1d
lgw_receive:1184: [1 17]
Note: LoRa packet
lgw_receive:1165: FIFO content: 1 f8 0 7 61
lgw_receive:1184: [3 17]
Note: LoRa packet

##### 2018-02-21 10:44:46 GMT #####
### [UPSTREAM] ###
# RF packets received by concentrator: 2
# CRC_OK: 100.00%, CRC_FAIL: 0.00%, NO_CRC: 0.00%
# RF packets forwarded: 2 (48 bytes)
# PUSH_DATA datagrams sent: 0 (0 bytes)
# PUSH_DATA acknowledged: 0.00%
### [DOWNSTREAM] ###
# PULL_DATA sent: 0 (0.00% acknowledged)
# PULL_RESP(onse) datagrams received: 0 (0 bytes)
# RF packets sent to concentrator: 0 (0 bytes)
# TX errors: 0
### BEACON IS DISABLED!
### [JIT] ###
# INFO: JIT queue contains 0 packets.
# INFO: JIT queue contains 0 beacons.
### GPS IS DISABLED!
### [PERFORMANCE] ###
# Upstream radio packet quality: 100.00%.
# Semtech status report send.
##### END #####
```

[Next page](https://www.thethingsnetwork.org/forum/t/multitech-ttn-connection-lost-after-bad-connectivity/12355.md?page=2)