Hello,
we are running The Things Stack Open Source v3.34.1 on Linux (Docker).
Our deployment includes about 100 gateways (MikroTik and Kona Enterprise) and around 20,000 end devices — mainly Zenner, BMeter, and Maddalena water meters.
Recently, the entire stack suddenly stopped working because the server disk filled up.
After checking, we found that one Things Stack container log had grown to ~90 GB.
The log mostly contains repeated errors like these:
2025-11-11T13:16:00.588430361Z INFO Finished unary call {“duration”: 0.0003, “error”: “error:pkg/networkserver:duplicate_uplink (duplicate uplink)”, “error_correlation_id”: “1562a7e3a1c64285b4269b02f4f46977”, “error_name”: “duplicate_uplink”, “error_namespace”: “pkg/networkserver”, “grpc.method”: “HandleUplink”, “grpc.service”: “ttn.lorawan.v3.GsNs”, “grpc_code”: “AlreadyExists”, “namespace”: “grpc”, “peer.address”: “pipe”, “request_id”: “01K9SGY66B29DNVRHE0VD3XZQ7”}
2025-11-11T13:16:00.588587861Z INFO Finished unary call {“duration”: 0.0019, “error”: “error:pkg/networkserver:device_not_found (device not found)”, “error_cause”: “error:pkg/networkserver/redis:no_uplink_match (no device matches uplink)”, “error_correlation_id”: “ec79c7df09384c3c900d87ac94d3638b”, “error_name”: “device_not_found”, “error_namespace”: “pkg/networkserver”, “grpc.method”: “HandleUplink”, “grpc.service”: “ttn.lorawan.v3.GsNs”, “grpc_code”: “NotFound”, “namespace”: “grpc”, “peer.address”: “pipe”, “request_id”: “01K9SGY66AC7SXXRGBSZFAGJ3C”}
As we understand these are not critical errors, but I don’t if we should ignore them or try to solve them as there are no device or gateway ids in these logs. For now I cleared logs, but again, the file keeps growing. How could we approach this problem?