Deploy The Things Stack
 in your local network
TheThings Network

The Things Network Global Team

Posted on 15-06-2021

The Things Stack can be used for large-scale global deployments, such as through The Things Stack Community Edition (aka The Things Network) or The Things Stack commercial deployments, but it can also be used for small-scale local deployments, including networks without a connection to the public Internet.

The Things Stack now offers multi-architecture Docker builds, allowing the use of Docker to deploy The Things Stack on devices with ARM CPUs, such as the Raspberry Pi.

At The Things Stack conference on May 28, Hylke Visser, Lead Software Engineer at The Things Industries, demonstrated how to deploy The Things Stack Enterprise on a Raspberry Pi in your local network. Let’s get you started, follow this demo:

You can also watch the session from the conference.

What do you need?

  • Your computer (a MacBook in this demo)
  • Raspberry Pi
    MicroSD card
    A screen and keyboard (only for installation)
  • A Local Network (WiFi, but Ethernet can also be used)
    DHCP Server (that assigns a static IP address to your Raspeberry Pi)
    DNS Records or DNS Server in your local network (optional)
    Tip: Your DHCP server can automatically configure a local DNS server on clients
    Internet Connection (only for installation)
    Possibility to use Packet Broker: LoRa Alliance NetID, DavAddr block(s), API key (contact our experts if needed)
    The Things Stack Enterprise license key (installation without a license key possible to configure it later)

NOTE: Most of these instructions also work for the open source edition of The Things Stack.

Let’s get started

Install the Raspberry Pi Operating System: https://www.raspberrypi.org/software/
Download the Raspberry Pi Imager.
Start the Raspberry Pi Imager, select the Raspberry Pi OS Lite.
Select the SD card.
Click Write (you may have to enter your password).
When it's finished, take the SD card and slide it into your Raspberry Pi. The first time you power on your Raspberry Pi it may restart a couple of times, but eventually you'll see the login prompt.

Configure WiFi and enable SSH, so that you can login to the Raspberry Pi from the computer, and copy-paste some commands instead of having to type everything yourself. To do so, follow these steps:
Login with username pi and password raspberry.
Run sudo raspi-config to start the configuration tool.
$ sudo raspi-config

To set up WiFi:
Go to System Options and then Wireless LAN.
Find and select your country in the list.
Enter the SSID of your WiFi network and the password.

To enable SSH:
Go to Interface Options and then SSH.
Enable the SSH server and change the password of the pi user.
Pess tab and select Finish to reboot the Raspberry Pi.
When you see the login prompt again, you'll now also see a line that says My IP address is and then your IP address. In this demo, it's 192.168.178.43.

Back on the Mac, use SSH to login to the Raspberry Pi:
Type ssh pi@192.168.178.43 and press enter.
$ ssh pi@192.168.178.43
Enter the password (if you didn't already change it, it's still raspberry).

Make sure everything is up-to-date:
Run apt-get update.
$ sudo apt-get update

This is going to take some time, but when it's finished, you can install Docker.

For the Docker installation, follow the Debian instructions on the Docker website:
- https://docs.docker.com/engine/install/debian
- https://docs.docker.com/engine/install/linux-postinstall

First download Docker's GPG key.
$ curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
Then we add their apt repository.
$ echo \ "deb [arch=armhf signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian \ $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
Again we apt-get update.
$ sudo apt-get update
And now we'll apt-get install the Docker components; we are also going to install docker-compose.
$ sudo apt-get install docker-ce docker-ce-cli containerd.io docker-compose
When this is all done, we're going to make sure our Pi user can run Docker containers.
We create the Docker group.
$ sudo groupadd docker
Then we add the current user to that group.
$ sudo usermod -aG docker $USER
And load it.
$ newgrp docker
Now, if we run docker ps, we should not get any errors.
$ docker ps

Next, we'll install The Things Stack:
We're going create a directory where we can keep all the files.
We'll then download the example docker-compose file, and make some changes for our specific deployment.
In the demo, Hylket put everything in /app/the-things-stack, but if you want to put it somewhere else, perhaps on an SSD that you attached to your Raspberry Pi, that's fine too.
$ sudo mkdir -p /app/the-things-stack $ sudo chown pi:pi /app/the-things-stack $ cd /app/the-things-stack
This is how we download the example docker-compose file from the documentation site, and then we'll open it for editing.
$ curl -sSL -o docker-compose.yml https://www.thethingsindustries.com/docs/getting-started/installation/configuration/docker-compose-enterprise.yml $ vi docker-compose.yml

NOTE: If you're installing the open source edition of The Things Stack, use docker-compose-open-source.yml.

First we'll change the version to 3, because the docker-compose version that we just installed doesn't like 3.7.
We'll comment out everything that mentions CockroachDB, because Cockroach doesn't have multi-architecture Docker images yet, so it won't work on your Raspberry Pi.
Instead we'll uncomment PostgreSQL.
We want to use Postgres 12, Redis 6 and The Things Stack 3.13.0.

NOTE: On https://www.thethingsindustries.com/docs/whats-new/ you can check if there are newer versions of The Things Stack.

We'll also comment out anything related to TLS because we'll not use that in this deployment.
Now we can download Postgres, Redis and The Things Stack by running docker-compose pull.

Time to configure The Things Stack:
For the hostname you can use the IP address of the Raspberry Pi, but in the demo, Hylke prefered to use domain names, so I set up demo.thethingsconference.com in the DNS server in my local network.
$ HOSTNAME=192.168.178.43 $ HOSTNAME=demo.thethingsconference.com
We'll generate random secrets for the Console, and for the Device Claiming Server.
$ CONSOLE_SECRET=$(openssl rand -hex 16) $ CLAIM_SECRET=$(openssl rand -hex 16)

NOTE: You don't need CLAIM_SECRET when installing the open source edition of The Things Stack.

Hylke keeps network-wide options in the config file, and cluster-specific options in the environment.
So we'll create an empty config file that we'll edit later, and set some environment variables for our cluster.
$ mkdir -p config/stack $ touch config/stack/ttn-lw-stack-docker.yml
We'll first have The Things Stack write a full list of environment variables to a file, and then I'll create an empty file so that we can cherry-pick the environment variables we want to override.
$ docker-compose run --rm stack config --env > stack.default.env $ touch stack.env
First, we're going to disable the TLS listeners.
We search for LISTEN_TLS, and replace those addresses with an empty string.
$ grep LISTEN_TLS stack.default.env | sed -e "s/:888[0-9]//" >> stack.env
Then we search for the PUBLIC_TLS_ADDRESS and clear those out too.
$ grep PUBLIC_TLS_ADDRESS stack.default.env | sed -e "s/localhost:888[0-9]//" >> stack.env
Next we're going to change all localhost addresses to our hostname.
We'll start with the MQTT ports that run on port 1881, 1882 and 1883.
$ grep localhost:188[123] stack.default.env | sed -e "s/localhost/$HOSTNAME/" >> stack.env
Our HTTP server runs on port 80, so in addition to changing localhost to our hostname, we can also remove that 1885.
$ grep localhost:1885 stack.default.env | sed -e "s/localhost:1885/$HOSTNAME/" >> stack.env
Then there are some options that use HTTPS, WSS or MQTT, so there we remove the s and change the 888 to 188.
$ grep 's://localhost:888' stack.default.env | sed -e "s|s://localhost:888|://$HOSTNAME:188|" >> stack.env
We'll also write the OAuth client secrets to the environment file.
$ echo TTN_LW_CONSOLE_OAUTH_CLIENT_SECRET=$CONSOLE_SECRET >> stack.env $ echo TTN_LW_DCS_OAUTH_CLIENT_SECRET=$CLAIM_SECRET >> stack.env

NOTE: You don't need TTN_LW_DCS_OAUTH_CLIENT_SECRET when installing the open source edition of The Things Stack.

And for the HTTP Cookie secrets, we'll just write some random values.
$ echo TTN_LW_HTTP_COOKIE_BLOCK_KEY=$(openssl rand -hex 32) >> stack.env $ echo TTN_LW_HTTP_COOKIE_HASH_KEY=$(openssl rand -hex 64) >> stack.env
Finally, we'll write our license key for The Things Stack Enterprise.
$ echo TTN_LW_LICENSE_KEY=CpYBChQKEmNvbmZlcm... >> stack.env

NOTE: You don't need TTN_LW_LICENSE_KEY when installing the open source edition of The Things Stack.
Now we have to quickly remove the quotes from the environment file, because docker-compose doesn't like those.
$ sed -i'' -e 's/"//g' stack.env
We don't need this file anymore.
$ rm stack.default.env
And before we continue, let's take a quick look at the environment file to make sure everything is right.
$ cat stack.env
We still need to make Docker Compose aware of this environment file.
Let's go back to the docker compose file, find the environment of the stack, and set our environment file as env_file.
$ vi docker-compose.yml

If your local deployment is not completely disconnected from the public internet, it's also nice to connect it to Packet Broker:
We'll write some options in the config file, and some others in the environment file.
In the config file we'll write the options that we would re-use in a different cluster that we might want to add to our network and in the environment file we'll add environment for this specific cluster.

Let's start with the config file.
$ vi config/stack/ttn-lw-stack-docker.yml
We'll create a PBA section, for the Packet Broker Agent, where we will configure the data plane address.
We're using the NetID of The Things Industries, and our tenant ID is htdvisser-edge.
We'll enable our deployment as a forwarder, and as a home network.
Next, we'll tell the Gateway Server to forward all traffic to both the local cluster, and to Packet Broker.
And we need to configure the Network Server to issue device addresses from our NetID and our device address prefix.
In the environment file we need to add our Cluster ID.
$ echo TTN_LW_PBA_CLUSTER_ID=rpi-1 >> stack.env
The Client ID is the Key ID of our Packet Broker API Key.
$ echo TTN_LW_PBA_OAUTH2_CLIENT_ID=OBQ4O2LBOOR4FUVU >> stack.env
And the Client Secret is the Secret Key.
$ echo TTN_LW_PBA_OAUTH2_CLIENT_SECRET=WCNJ7CJCOBSKAQXC... >> stack.env

Now let's initialize The Things Stack:
We're going to initialize the database, create a tenant, an admin user, and OAuth clients for the command-line interface, the Console and the Device Claiming Server.
Here's how we initialize the database of the Identity Server.
$ docker-compose run --rm stack is-db init
Then we create the default tenant for our deployment.
$ docker-compose run --rm stack is-db create-tenant

NOTE: You should skip this when installing the open source edition of The Things Stack.

Now we create the admin user.
$ docker-compose run --rm stack is-db create-admin-user \ --id admin \ --email your@email.com
It asks us to enter a password, and to repeat it.
Next is the OAuth client for the command-line interface.
$ docker-compose run --rm stack is-db create-oauth-client \ --id cli \ --name "Command Line Interface" \ --owner admin \ --no-secret \ --redirect-uri "local-callback" \ --redirect-uri "code"
The OAuth client for the Console.
$ docker-compose run --rm stack is-db create-oauth-client \ --id console \ --name "Console" \ --owner admin \ --secret "${CONSOLE_SECRET}" \ --redirect-uri "/console/oauth/callback" \ --logout-redirect-uri "/console"
And the OAuth client for the Device Claiming Server.
$ docker-compose run --rm stack is-db create-oauth-client \ --id device-claiming \ --name "Device Claiming Server" \ --owner admin \ --secret "${CLAIM_SECRET}" \ --redirect-uri "/claim/oauth/callback" \ --logout-redirect-uri "/claim"

NOTE: You should skip this when installing the open source edition of The Things Stack.

Time to start The Things Stack:
We run docker-compose up.
$ docker-compose up -d stack
And watch the logs.
$ docker-compose logs -f stack
We can see that The Things Stack is setting up the different components.
And now it's ready, listening for connections.
We can also see some Packet Broker traffic coming in already.
Now, let's see how we can access the Console of our local deployment.
In his browser, Hylke went to demo.thethingsconference.com, but if he hadn't set up DNS, that would have been the IP address of the Raspberry Pi.
We are going to login as the admin user that we just created, also with the password that we just entered..
Now let's see how to configure the command-line interface for this deployment.
We'll open a new tab in the terminal -- so this is now on the Mac, not the SSH session to the Raspberry Pi.
We type tti-lw-cli use demo.thethingsconference.com. And we set the gRPC port to 1884.
$ ttn-lw-cli use demo.thethingsconference.com --grpc-port 1884

NOTE: It's ttn-lw-cli if you're using the open source edition of The Things Stack.

We have to make two more changes to make sure that the CLI connects without TLS, so let's open up the config file for editing.
$ vi .ttn-lw-cli.yml
Here, we need to change HTTPS to HTTP.
And here we need to set insecure to true.
Now let's try to login.
We type tti-lw-cli login.
$ tti-lw-cli login

NOTE: It's ttn-lw-cli if you're using the open source edition of The Things Stack.

This will automatically open the web browser, and since we are already logged in, it doesn't ask for the username and password again.
It immediately tells me that the CLI successfully got an access token.
We can now close my browser window, and back in the terminal we'll see that the CLI indeed got that access token.
And that's how you install and configure a local deployment of The Things Stack on a Raspberry Pi.

Now you can register some gateways and end devices, and connect them to your local deployment.
With the Packet Broker integration, your local network can now also connect to your tenant on The Things Stack Cloud, and contribute coverage to The Things Stack Community Edition. You can see the Packet Broker documentation for more information on that.

Watch the demo:

Get started with The Things Stack using your The Things Network account
Scale with The Things Stack commercial deployments