LIG16 troubleshooting

Hi,

I am trying to setup my first gateway.

I have the LIG16 and I have followed the tutorial and things looks streight forward, but the GW has no connection in TTN.

Firmware: lgw-5.4.1610176959
Server addr: router.eu.thethings.network

Most important my “LoraWan process pkt_fwd” is not running, any chance to start it manually?

Some logs:

Without any information from the gateway such as system logs showing attempted packet forwarder start, it’s going to be hard for people here to help, and you may need to pursue support from Dragino.

If you can see a syslog / logread log then you may be able to see if for example the packet forwarder failed to start because of a configuration problem, or if it started up but then quit because it wasn’t getting protocol acks from the TTN server you tried to point it at.

I have attached some logs. I actually have contacted dragino already, no response yet.

Some key observations:

  • during the logged interval, your gateways has not received a single valid packet. It’s typically very hard to figure out what is going on unless there are packets flowing from legitimate nodes, you may need to rig one to be a bit more active than usual while debugging a gateway.

  • Your log seems to indicate an SX1302; it’s possible that this would cause any actual signal reports (not that you have any, but if you did) to trigger the bug with TTN not accepting the temperature value as floating point the way the reference gateway code encodes it.

  • For the most part, neither your pull nor push requests are being ack’d which suggests you are pointed at the wrong server or have Internet or DNS issues. However at one point your pull requests were all but completely ack’d while your pushes (apparently consisting only of stats packets) were not; this raised the question if what you are seeing is spotty connectivity/DNS, or instead/also a protocol compatibility issue.

This suggests you are triggering the issue mentioned in point 2. It would be better to start using V3 of the stack in stead of the (to be retired later this year) V2.
Connect to eu1.cloud.thethings.network, log in using the ‘Things ID’ (use the same credentials you use for V2) and add the gateway over there using either the LPS8 instructions or the more generic UDP forwarder instructions.

Interesting, review of (the base Semtech version of) the code shows the stats packets also include the temperature.

However in the log there’s only one instance of the pull requests being mostly acked; all the other cycles show it completely unacked.

We may be looking at both a TTN protocol bug and an Internet connectivity issue.

1 Like

Yep, the LIG16 is Dragino’s 1st SX1302 GW; awaiting sample unit for 1st tests! :wink:

Unfortunate model number; yes, the SX1302 has 16 decoders, but it only monitors the same number of frequencies as a traditional “8 channel” gateway.

A true “16 channel” gateway would have two concdentrator chips supported by four radios.

But anyway, the SX1302 is nice enough with honest specs.

Yes, suspect someone playing Marketing :rofl: though additional bank of receivers - even limited to up to SF10 (this is done for economics - each SF step I suspect effectively doubles on chip memory requirement to support dsp) adds greater capacity and typically density of lower SF nodes is higher so useful in real world and also I think whole chip good for US market where SF >10 would cause problems with dwell time etc., add that to other capabilities and its a nice chip though now effectively redundant in the face of the Sx1303 (with e.g. finer timing capabilities for geoloc etc.). I think of it more as 16 (LoRaWAN) signals/messages vs 16 (rf) channels given additional demod capacity - two overlapping SF7’s or SF’8s etc could be handled if I understand overviews I have seen, yieding higher network uplink capacity.

I also have a LIG16 and I’d confirm that it currently has a (partially) broken packet forwarder. Status uplinks are never received at TTN, but packets from an actual node are visible in the console.

Some other observations I have made so far:

  • DO NOT UNINSTALL ‘haserl-ui’! Even though the installed version and the one available in the repository have are the same version, they have different content. You have to factory reset to get it working again.
  • It looks like there is a Basics Station included. The filename matches and the helpmessage is similar to the one in the docs. There is also a CUPS server setting metioned in the AWS IoT settings page.
  • The default network settings are not what I would have expected. This is an end device (as in not a network router) but there are two firewall zones and they are both used. This should not matter for normal usage and is not used to hide anything from the user, I just don’t like it.

The orthogonality hasn’t meaningfully changed; what has is that they have more decoder engines to throw at a very busy (or potentially misleading-from-interference) set of sub-IF’s. In theory that makes it slightly more able to leverage the orthagonality; in practice I doubt it matters. Though it occurs to me I could probably engineer a stress test comparison…

Really what seems to have happened is that they went to a more modern silicon process where they could fit more logic at drastically less power - the lower operating voltage is a hint of that too.

My clients dream of solar, so we’re going in the 1302 direction for power alone… question is if we try to squeeze it all on an MCU to cut power there. One of these days I’m going to see if I can underclock the Linux SOC, being a packet forwarder is pretty low intensity.

1 Like

Is this an an AR9331 OpenWRT box like there others? Could you build a standard packet forwarder yourself? The sx1302 code may take some adjustment, though I don’t recall if the AR9331 has SPI issues needing a half duplex modification, or if that’s only its Mediatek competitor MT7628.

To the extent that the issue may be the TTN floating point temperature bug, I bet you could fix that even without a rebuild by hexediting the binary to change the "temp" in the json sprintf’s to "xxxx" presumably causing that field to simply be ignored - make sure to get both the stats and rxpk versions (probably end up doing the debug output ones too)

Yes, it’s an OpenWRT box, /proc/cpuinfo tells me it’s an “Atheros AR9330 rev 1”. I could probably compile a packet forwarder myself, just have to get a cross-compiler setup ready. I’ll try hexediting the binary tomorrow.

If anyone wants to try binary patching the packet forwarder executable to “hide” the temperature field that TTN has issues with by renaming it, you could try this on the executable. Some gatways may have a sed otherwise you’d have to copy it off, make the change and then put it back.

sed 's/\"temp\"/\"xxxx\"/g' original_packet_forwarder > hacked_packet_forwarder

should I run this?

I hadn’t uninstalled anything, I just bootet the gateway and hoped this to work :slight_smile:

Can you give some hints where to look?

If you’re familiar with command line operations on OpenWrt and know that your particular box has an overlay filesystem that can be reverted and know where the packet forwarder executable is, potentially.

Otherwise probably not

Then you have not done anything wrong. My observations are mainly there if you are trying do look under the hood of the LIG16. The best way to see if the gateway is forwarding packets is to use an actual end node.

This unfortunately did not change anything, however there doesn’t seem to be a mention of “temp” anywhere in the binary:

root@ttngw01:~# strings /usr/bin/fwd | grep temp
lgw_get_temperature
# Concentrator temperature: %.0f C #

Interesting. Can you packet sniff the traffic on the box or route it through something that can?

I guess another debug approach would be to a point it at a copy of chirpstack gateway bridge to satisfy the UDP protocol, and then examine the MQTT or websocket output thereof.

Or I guess you could run your own private instance of the TTN stack to get full debug access, but that’s getting a bit determined.