V3 TTN - Scheduled Maintenance

link to this topic

Just a heads up for TTN users who regularly use the Forum but who may not follow on Slack ops channel or view the TTN Status pages.

A small but significant rollup update and series of fixes for the V3 TTN instance is scheduled for April 12 and may lead to some service interuption so take this message as a heads up! :slight_smile:


Scheduled Maintenance

Upgrade The Things Network to v3.12.0 Apr 12, 11:00-13:00 CEST

During this maintenance window we will upgrade The Things Network to v3.12.0

We do expect some short service interruptions during this update.

Here is the changelog since the current version v3.11.3:


  • API and CLI commands for listing, searching for and restoring recently deleted applications, OAuth clients, gateways, organizations and users.
  • State Description fields for adding context to the (rejected, flagged, suspended) state of Users, OAuth clients and tenants.
  • This requires a database schema migration (ttn-lw-stack is-db migrate) because of the added columns.
  • Configure retention policy for the Storage Integration database (only supported when using TimescaleDB).
  • You will need to run the ttn-lw-stack storage-db migrate command to apply the new configuration. See the --timescaledb.chunk-time-interval, --timescaledb.enable-retention-policy and --timescaledb.retention-days command-line flags.
  • Searching for gateways by EUI.
  • Searching for users and OAuth clients by state.
  • Gateway Server forwards Tx Acknowlegdment packets to the Network Server for scheduled downlinks. These can be used by the Network Server to forward downlink_ack upstream messages to the Application Server.
  • UDP connection error caching. The duration can be configured via the gs.udp.connection-error-expires configuration entry.
  • Option to require individual gateways to use authenticated connections.
  • This requires a database schema migration (ttn-lw-stack is-db migrate) because of the added columns.
  • Login Tokens (magic login links) that can be used for password-less login.
  • This requires a database schema migration (ttn-lw-stack is-db migrate) because of the added columns.
  • This feature is disabled by default; use the new is.login-tokens.enabled option to enable it.
  • Packet Broker registration, configuration of routing policies, listing home networks and viewing routing policies set by forwarding networks. See ttn-lw-cli packetbroker --help for more information.
  • Support LoRa 2.4 GHz with Packet Broker.
  • Include gateway identifiers from Packet Broker in metadata.
  • Support for claiming gateways through CLI. See ttn-lw-cli gateways claim --help for more information.
  • Session and MAC state import functionality. This means that devices can be migrated without rejoin.
  • Rate limiting for HTTP endpoints, gRPC endpoints, MQTT, UDP and WebSockets connections.
  • Rate limiting is disabled by default. Refer to the rate-limiting configuration entry to enable.
  • Profile settings link to header dropdown menu.


  • Changed the pub/sub channels that the Redis backend of the Events system uses.
  • Changed the encoding of events transported by the Redis backend of the Events system.
  • All external HTTP calls are now using TLS client configuration. This fixes issues where HTTP calls would fail if custom (e.g. self-signed) CAs were used.
  • All external HTTP calls are now using a default timeout. This fixes issues where HTTP calls would stall for a long time.
  • All value wrappers now are encoded and decoded as the value being wrapped in JSON. That means, that, e.g. format of mac_settings.rx1_delay is changed from {"value": 2} to just 2.
  • Changed the error that is returned when attempting to validate already validated contact info.
  • This requires a database schema migration (ttn-lw-stack is-db migrate) because of the added column.
  • Update Go to 1.16
  • Network Server now performs more strict validation and will disallow creation and updates of invalid devices.
  • DevEUI is not required for multicast devices anymore, regarding of LoRaWAN version.


  • AWS IoT Integration as pub/sub integration. The integration is only available as application package.
  • This requires a database migration (ttn-lw-stack as-db migrate).


  • Incorrect documentation url for event details data formats.
  • Search functionality for applications, gateways and organizations in the Console.
  • Error handling of end device template formats for the application overview page in the Console.
  • Payload size limits for AU915 data rates 8 and 9, which are now consistent with Regional Parameters RP002-1.0.2.
  • Payload size limit calculation in Network Server.
  • Occasional panic in Network Server on downlink with corrupted device states.
  • Occasional panic in Identity Server on extracting log fields from invalid requests.
  • Print an error message stating that the Storage Integration is not available in the open source edition of The Things Stack when trying to execute ttn-lw-stack storage-db commands.
    Posted on Apr 7, 17:23 CEST

Hmmm, Discourse should have stopped showing this as a banner on Monday afternoon. I’ll look at the settings it any other banners come along.

:+1: Thanks Hylke

I am interested whether this maintenance is still causing any know issues. I myself have encountered with two issues:

  1. “Host cluster failed to handle message” is an answer to every uplink packet (including join requests), it seems.
  2. When I was trying to remove a gateway from one cloud instance and add to another, the gateway now never appears as connected.
    The same setup was working just fine earlier. Unfortunately, I have not seen any information about possible issues anywhere.

A post was split to a new topic: TTB v3 Scheduled Maintenance - Monday 3rd May

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.