TheThingsStack V3.8.7 with self-signed certificate -> Forbidden Token exchange refused

Hello, everyone,

Unfortunately all older forum entries dealing with this topic are closed, so I have opened a new entry here. I would like to use TheThingsStack V3.8.7 with self-signed certificates. To do so I followed the installation guide. Nevertheless for me it was still necessary to update the docker-compose.yml file because otherwise, for example, after logging into the web console I was not redirect to the console. Instead the url just stayed at console/oauth.

Below are my configuration file:

version: '3.7'
services:
  stack:
    image: 'thethingsnetwork/lorawan-stack:latest' 
    entrypoint: 'ttn-lw-stack -c /config/stack/ttn-lw-stack-docker.yml'
    command: 'start'
   restart: 'unless-stopped'
    volumes:
      - './blob:/srv/ttn-lorawan/public/blob'
      - './config/stack:/stack:ro'
      - './config:/config:ro'        
    environment:      
      TTN_LW_TLS_CERTIFICATE: /config/cert.pem
      TTN_LW_CA: /config/cert.pem
      TTN_LW_TLS_KEY: /config/key.pem
      TTN_LW_CONSOLE_OAUTH_AUTHORIZE_URL: "http://sub.mydomain.com/oauth/authorize"
      TTN_LW_CONSOLE_OAUTH_TOKEN_URL: "http://sub.mydomain.com/oauth/token"
      TTN_LW_IS_OAUTH_UI_CANONICAL_URL: "http://sub.mydomain.com/oauth"
      TTN_LW_BLOB_LOCAL_DIRECTORY: '/srv/ttn-lorawan/public/blob'
      TTN_LW_REDIS_ADDRESS: 'my-dedicated-redis-server:6379'      
      TTN_LW_IS_DATABASE_URI: 'postgres://root@my-dedicated-cockroachdb:26257/ttn_lorawan?sslmode=disable'
      TTN_LW_LOG_LEVEL: debug
    ports:      
      - '80:1885'
      - '443:8885'     
      - '1881:1881'
      - '8881:8881'     
      - '1882:1882'
      - '8882:8882'      
      - '1883:1883'
      - '8883:8883'
      - '1884:1884'
      - '8884:8884'
      - '1887:1887'
      - '1700:1700/udp'

And here the content of my /config/stack/ttn-lw-stack-sdocker.yml:

is:
  email:
    sender-name: 'The Things Stack'
    sender-address: 'noreply@sub.mydomain.com'
    network:
      name: 'The Things Stack'
      console-url: 'https://sub.mydomain.com/console'
      identity-server-url: 'https://sub.mydomain.com/oauth'
  oauth:
    ui:
      canonical-url: 'https://sub.mydomain.com/oauth'
      is:
        base-url: 'https://sub.mydomain.com/api/v3'

http:
  cookie:
    block-key: '' # In the actual file this string is not empty
    hash-key: '' # In the actual file this string is not empty
  metrics:
    password: '' # In the actual file this string is not empty
  pprof:
    password: '' # In the actual file this string is not empty

#tls:
#   source: file
#   root-ca: /run/secrets/cert.pem
#   certificate: /run/secrets/cert.pem
#   key: /run/secrets/key.pem

gs:
  mqtt:
    public-address: 'sub.mydomain.com:1882'
    public-tls-address: 'sub.mydomain.com:8882'
  mqtt-v2:
    public-address: 'sub.mydomain.com:1881'
    public-tls-address: 'sub.mydomain.com:8881'

gcs:
  basic-station:
    default:
      lns-uri: 'wss://sub.mydomain.com:8887'
  the-things-gateway:
    default:
      mqtt-server: 'mqtts://sub.mydomain.com:8881'

console:
  ui:
    canonical-url: 'https://sub.mydomain.com/console'
    is:
      base-url: 'https://sub.mydomain.com/api/v3'
    gs:
      base-url: 'https://sub.mydomain.com/api/v3'
    ns:
      base-url: 'https://sub.mydomain.com/api/v3'
    as:
      base-url: 'https://sub.mydomain.com/api/v3'
    js:
      base-url: 'https://sub.mydomain.com/api/v3'
    qrg:
      base-url: 'https://sub.mydomain.com/api/v3'
    edtc:
      base-url: 'https://sub.mydomain.com/api/v3'

  oauth:
    authorize-url: 'https://sub.mydomain.com/oauth/authorize'
    logout-url: 'https://sub.mydomain.com/oauth/logout'
    token-url: 'https://sub.mydomain.com/oauth/token'
    client-id: '' # In the actual file this string is not empty
    client-secret: '' In the actual file this string is not empy

Here is the log that comes after I log into the console:

stack_1  |  DEBUG Run database query                       duration=4ms namespace=identityserver query=SELECT users.id, users.created_at, users.updated_at, password FROM "users" LEFT JOIN accounts ON accounts.account_type = $1 AND accounts.account_id = users.id WHERE "users"."deleted_at" IS NULL AND ((accounts.uid = $2)) ORDER BY "users"."id" ASC LIMIT 1 rows=1 source=user_store.go:121 values=[user christophmerscher]
stack_1  |  DEBUG Run database query                       duration=2ms namespace=identityserver query=SELECT * FROM "accounts"  WHERE "accounts"."deleted_at" IS NULL AND (("account_id" IN ($1) AND "account_type" = $2)) ORDER BY "accounts"."id" ASC rows=1 source=user_store.go:121 values=[4af2f2d7-7ed7-4a62-9dd0-4a555ea5fc60 user]
stack_1  |  DEBUG Run database query                       duration=3ms namespace=identityserver query=SELECT users.id FROM "users" LEFT JOIN accounts ON accounts.account_type = $1 AND accounts.account_id = users.id WHERE "users"."deleted_at" IS NULL AND ((accounts.uid = $2)) ORDER BY "users"."id" ASC LIMIT 1 rows=1 source=store.go:59 values=[user christophmerscher]
stack_1  |  DEBUG Run database query                       duration=2ms namespace=identityserver query=INSERT INTO "user_sessions" ("created_at","updated_at","user_id","expires_at") VALUES ($1,$2,$3,$4) RETURNING "user_sessions"."id" rows=1 source=store.go:72 values=[2020-07-31 08:12:46.533 +0000 UTC 2020-07-31 08:12:46.533 +0000 UTC 4af2f2d7-7ed7-4a62-9dd0-4a555ea5fc60 <nil>]
stack_1  |   INFO Request handled                          duration=41.40977ms method=POST namespace=web remote_addr=128.141.150.246 request_id=01EEHYWFH0CC4B546YW2AV9V8B response_size=0 status=204 url=http://sub.mydomain.com/oauth/api/auth/login
stack_1  |  DEBUG Run database query                       duration=3ms namespace=identityserver query=SELECT users.id FROM "users" LEFT JOIN accounts ON accounts.account_type = $1 AND accounts.account_id = users.id WHERE "users"."deleted_at" IS NULL AND ((accounts.uid = $2)) ORDER BY "users"."id" ASC LIMIT 1 rows=1 source=store.go:59 values=[user christophmerscher]
stack_1  |  DEBUG Run database query                       duration=2ms namespace=identityserver query=SELECT * FROM "user_sessions"  WHERE ("user_sessions"."id" = $1) AND ("user_sessions"."user_id" = $2) rows=1 source=user_session_store.go:88 values=[e9e06a41-6a71-48aa-82d8-e751d6376eea 4af2f2d7-7ed7-4a62-9dd0-4a555ea5fc60]
stack_1  |  DEBUG Run database query                       duration=2ms namespace=identityserver query=SELECT * FROM "clients"  WHERE "clients"."deleted_at" IS NULL AND ((client_id = $1)) ORDER BY "clients"."id" ASC LIMIT 1 rows=1 source=client_store.go:113 values=[console]
stack_1  |  DEBUG Run database query                       duration=1ms namespace=identityserver query=SELECT * FROM "attributes"  WHERE ("entity_id" IN ($1) AND "entity_type" = $2) ORDER BY "attributes"."id" ASC rows=0 source=client_store.go:113 values=[7c147cb8-6ce7-443f-8e14-298e1fffc806 client]
stack_1  |  DEBUG Run database query                       duration=2ms namespace=identityserver query=SELECT clients.id FROM "clients"  WHERE "clients"."deleted_at" IS NULL AND ((client_id = $1)) ORDER BY "clients"."id" ASC LIMIT 1 rows=1 source=store.go:59 values=[console]
stack_1  |  DEBUG Run database query                       duration=2ms namespace=identityserver query=SELECT users.id FROM "users" LEFT JOIN accounts ON accounts.account_type = $1 AND accounts.account_id = users.id WHERE "users"."deleted_at" IS NULL AND ((accounts.uid = $2)) ORDER BY "users"."id" ASC LIMIT 1 rows=1 source=store.go:59 values=[user christophmerscher]
stack_1  |  DEBUG Run database query                       duration=2ms namespace=identityserver query=SELECT * FROM "client_authorizations"  WHERE ("client_authorizations"."client_id" = $1) AND ("client_authorizations"."user_id" = $2) ORDER BY "client_authorizations"."id" ASC LIMIT 1 rows=1 source=oauth_store.go:104 values=[7c147cb8-6ce7-443f-8e14-298e1fffc806 4af2f2d7-7ed7-4a62-9dd0-4a555ea5fc60]
stack_1  |  DEBUG Run database query                       duration=2ms namespace=identityserver query=UPDATE "client_authorizations" SET "created_at" = $1, "updated_at" = $2, "client_id" = $3, "user_id" = $4, "rights" = $5  WHERE "client_authorizations"."id" = $6 rows=1 source=oauth_store.go:116 values=[2020-07-30 11:01:36.034 +0000 UTC 2020-07-31 08:12:46.564 +0000 UTC 7c147cb8-6ce7-443f-8e14-298e1fffc806 4af2f2d7-7ed7-4a62-9dd0-4a555ea5fc60 {[RIGHT_ALL] {} 0} 77d48799-7cfb-4e30-9c47-674b3749f0e3]
stack_1  |  DEBUG Run database query                       duration=2ms namespace=identityserver query=SELECT clients.id FROM "clients"  WHERE "clients"."deleted_at" IS NULL AND ((client_id = $1)) ORDER BY "clients"."id" ASC LIMIT 1 rows=1 source=store.go:59 values=[console]
stack_1  |  DEBUG Run database query                       duration=2ms namespace=identityserver query=SELECT users.id FROM "users" LEFT JOIN accounts ON accounts.account_type = $1 AND accounts.account_id = users.id WHERE "users"."deleted_at" IS NULL AND ((accounts.uid = $2)) ORDER BY "users"."id" ASC LIMIT 1 rows=1 source=store.go:59 values=[user christophmerscher]
stack_1  |  DEBUG Run database query                       duration=1ms namespace=identityserver query=INSERT INTO "authorization_codes" ("created_at","updated_at","client_id","user_id","user_session_id","rights","code","redirect_uri","state","expires_at") VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10) RETURNING "authorization_codes"."id" rows=1 source=store.go:72 values=[2020-07-31 08:12:46.558 +0000 UTC 2020-07-31 08:12:46.573 +0000 UTC 7c147cb8-6ce7-443f-8e14-298e1fffc806 4af2f2d7-7ed7-4a62-9dd0-4a555ea5fc60 0xc00030e6a8 {[RIGHT_ALL] {} 0} MF2XI.AJO7SWWNJVCTU57EASTJBJNE7FAXKLRTB3XVBJA.YU7CTA7QY3KNY6UPKTWF5E545453MIZZILRJHK4WGBXPEP6OVFEQ https://sub.mydomain.com/console/oauth/callback GVuTmwbLcCbU7K-v 2020-07-31 08:17:46.558817984 +0000 UTC]
stack_1  |   INFO Request handled                          duration=26.231695ms method=GET namespace=web remote_addr=128.141.150.246 request_id=01EEHYWFJPP14BHM6V3HR0358D response_size=0 status=302 url=http://sub.mydomain.com/oauth/authorize?client_id=console&redirect_uri=https%3A%2F%2Fsub.mydomain.com%2Fconsole%2Foauth%2Fcallback&response_type=code&state=GVuTmwbLcCbU7K-v
stack_1  |   INFO Client error                             duration=1.004904987s method=GET namespace=web remote_addr=128.141.150.246 request_id=01EEHYWFM3CG5B8W6FD1DK3N49 response_size=940 status=403 url=https://sub.mydomain.com/console/oauth/callback?code=MF2XI.AJO7SWWNJVCTU57EASTJBJNE7FAXKLRTB3XVBJA.YU7CTA7QY3KNY6UPKTWF5E545453MIZZILRJHK4WGBXPEP6OVFEQ&state=GVuTmwbLcCbU7K-v
stack_1  |   INFO Request handled                          duration=5.058578ms method=GET namespace=web remote_addr=128.141.150.246 request_id=01EEHYWGMSHP8BBMDBK29BEEXE response_size=273513 status=200 url=https://sub.mydomain.com/assets/console.c6ddc5f8af3f8c39f150.css
stack_1  |   INFO Request handled                          duration=35.401639ms method=GET namespace=web remote_addr=128.141.150.246 request_id=01EEHYWGMS6VQAWMCKQT836SK1 response_size=3844883 status=200 url=https://sub.mydomain.com/assets/console.df2eb219cc37cf35b89a.js
stack_1  |   INFO Client error                             duration=570.165µs method=GET namespace=web remote_addr=128.141.150.246 request_id=01EEHYWH4H70VJXT06P6KBBDCR response_size=198 status=401 url=https://sub.mydomain.com/console/api/auth/token
stack_1  |   INFO Request handled                          duration=506.619µs method=GET namespace=web remote_addr=128.141.150.246 request_id=01EEHYWH4SERGY7QCZGN7ADKWP response_size=35390 status=200 url=https://sub.mydomain.com/assets/console-touch-icon.png
stack_1  |   INFO Request handled                          duration=318.424µs method=GET namespace=web remote_addr=128.141.150.246 request_id=01EEHYWH4SVWCBQ750R2S7F6CV response_size=6239 status=200 url=https://sub.mydomain.com/assets/console-favicon.svg
stack_1  |   INFO Request handled                          duration=300.513µs method=GET namespace=web remote_addr=128.141.150.246 request_id=01EEHYWH575J9ANF2JMW5CJ4KH response_size=5004 status=200 url=https://sub.mydomain.com/assets/logo.svg
stack_1  |   INFO Request handled                          duration=3.859516ms method=GET namespace=web remote_addr=128.141.150.246 request_id=01EEHYWH5M92MBB176Z8PD3360 response_size=699912 status=200 url=https://sub.mydomain.com/assets/ttn-console-bg.498252edc489187693a4e32162260925.png

Please not that in my actual file instead of sub.mydomain.com I use a domain that belongs to me.

I already read a post entry with a simillar problem but this did not do the trick for me. Also I checked this github issue where it is mentioned that it have been solved. However, this is not the case for me.

Any ideas ?

No one ?

Okay in case anyone facing the same issue. For me it was that when I was being redirected to the console it looked for the certificates in a different place. Copying the certificates to this place solved the issue.

2 Likes

Hi, could you expand a bit more on the solution? Where did you have to copy the certificates? Did you find out why it’s searching for them somewhere else?

Hi,

I did not find out why thethingsstack was searching for certificates on a different location, however I can say that this only happened with TTN 3.8.7 with TTN less or bigger as this version I did not have this issue.

For me the issue was that I had store my certificates in config/certificates but TTN was looking at config/.

Moving them to this folder did the trick. However, I would like to raise again that this was only the case for 3.8.7. When I used the same config on different versions this did not happen.

Thanks for the info. It seems like it’s not the same cause but I can assume now that it’s a certificate problem of some sort. I’ll keep investigating this.

Once again, thanks!

Maybe I can help you a bit further if you copy the error messages that you have here.

Sure, let’s try. Thanks!

From the browser console I can see a network package giving me an error when it tries to redirect me after logging in.

Request URL: https://redacted/console/oauth/callback?code=a_long_code&state=i0FSI5ummAkAUYPk
Status code: 403

This is the entire stack log from start to Token exchange refused:

stack_1 | INFO Setting up core component
stack_1 | WARN No cookie hash key configured, generated a random one hash_key=REDACTED namespace=web
stack_1 | WARN No cookie block key configured, generated a random one block_key=REDACTED namespace=web
stack_1 | DEBUG Loaded manifest.yaml mount=/assets namespace=web path=/srv/ttn-lorawan/public
stack_1 | DEBUG Serving static assets mount=/assets namespace=web path=/srv/ttn-lorawan/public
stack_1 | INFO Setting up Identity Server
stack_1 | INFO Setting up Gateway Server
stack_1 | INFO Setting up Network Server
stack_1 | INFO Setting up Application Server
stack_1 | INFO Setting up Join Server
stack_1 | INFO Setting up Console
stack_1 | INFO Setting up Gateway Configuration Server
stack_1 | DEBUG Loaded TLS certificate
stack_1 | INFO Setting up Device Template Converter
stack_1 | INFO Setting up QR Code Generator
stack_1 | INFO Setting up Packet Broker Agent
stack_1 | INFO Starting…
stack_1 | DEBUG Initializing gRPC server…
stack_1 | DEBUG Starting loopback connection
stack_1 | DEBUG ccResolverWrapper: sending update to cc: {[{in-process 0 }] } namespace=grpc
stack_1 | DEBUG Channel switches to new LB policy “pick_first” namespace=grpc
stack_1 | DEBUG Subchannel Connectivity change to CONNECTING namespace=grpc
stack_1 | DEBUG Setting up gRPC gateway
stack_1 | DEBUG pickfirstBalancer: HandleSubConnStateChange: 0xc000a1a690, {CONNECTING } namespace=grpc
stack_1 | DEBUG Channel Connectivity change to CONNECTING namespace=grpc
stack_1 | DEBUG Subchannel picks a new address “in-process” to connect namespace=grpc
stack_1 | DEBUG Exposed services namespace=grpc services=[ttn.lorawan.v3.AppAs ttn.lorawan.v3.ApplicationAccess ttn.lorawan.v3.ApplicationPackageRegistry ttn.lorawan.v3.ApplicationPubSubRegistry ttn.lorawan.v3.ApplicationRegistry ttn.lorawan.v3.ApplicationWebhookRegistry ttn.lorawan.v3.As ttn.lorawan.v3.AsEndDeviceRegistry ttn.lorawan.v3.AsJs ttn.lorawan.v3.AsNs ttn.lorawan.v3.ClientAccess ttn.lorawan.v3.ClientRegistry ttn.lorawan.v3.Configuration ttn.lorawan.v3.ContactInfoRegistry ttn.lorawan.v3.EndDeviceQRCodeGenerator ttn.lorawan.v3.EndDeviceRegistry ttn.lorawan.v3.EndDeviceRegistrySearch ttn.lorawan.v3.EndDeviceTemplateConverter ttn.lorawan.v3.EntityAccess ttn.lorawan.v3.EntityRegistrySearch ttn.lorawan.v3.Events ttn.lorawan.v3.GatewayAccess ttn.lorawan.v3.GatewayRegistry ttn.lorawan.v3.Gs ttn.lorawan.v3.GsNs ttn.lorawan.v3.GsPba ttn.lorawan.v3.GtwGs ttn.lorawan.v3.Js ttn.lorawan.v3.JsEndDeviceRegistry ttn.lorawan.v3.Ns ttn.lorawan.v3.NsEndDeviceRegistry ttn.lorawan.v3.NsGs ttn.lorawan.v3.NsJs ttn.lorawan.v3.NsPba ttn.lorawan.v3.OAuthAuthorizationRegistry ttn.lorawan.v3.OrganizationAccess ttn.lorawan.v3.OrganizationRegistry ttn.lorawan.v3.UserAccess ttn.lorawan.v3.UserInvitationRegistry ttn.lorawan.v3.UserRegistry]
stack_1 | DEBUG Initializing cluster…
stack_1 | WARN No cluster key configured, generated a random one key=REDACTED
stack_1 | DEBUG Initializing web server…
stack_1 | DEBUG Subchannel Connectivity change to READY namespace=grpc
stack_1 | DEBUG pickfirstBalancer: HandleSubConnStateChange: 0xc000a1a690, {READY } namespace=grpc
stack_1 | DEBUG Channel Connectivity change to READY namespace=grpc
stack_1 | DEBUG Initializing interop server…
stack_1 | DEBUG Starting gRPC server…
stack_1 | DEBUG Creating listener address=:1884
stack_1 | INFO Listening for connections address=:1884 namespace=grpc protocol=gRPC
stack_1 | DEBUG Creating listener address=:8884
stack_1 | DEBUG Loaded TLS certificate
stack_1 | INFO Listening for connections address=:8884 namespace=grpc protocol=gRPC/tls
stack_1 | DEBUG Started gRPC server
stack_1 | DEBUG Starting web server…
stack_1 | DEBUG Creating listener address=:1885
stack_1 | INFO Listening for connections address=:1885 namespace=web protocol=Web
stack_1 | DEBUG Creating listener address=:8885
stack_1 | DEBUG Loaded TLS certificate
stack_1 | INFO Listening for connections address=:8885 namespace=web protocol=Web/tls
stack_1 | DEBUG Started web server
stack_1 | DEBUG Starting interop server
stack_1 | DEBUG Creating listener address=:8886
stack_1 | DEBUG Loaded TLS certificate
stack_1 | INFO Listening for connections address=:8886 namespace=interop protocol=Interop/tls
stack_1 | DEBUG Started interop server
stack_1 | DEBUG Joining cluster…
stack_1 | DEBUG Joined cluster
stack_1 | DEBUG Starting tasks
stack_1 | DEBUG Started tasks
stack_1 | DEBUG Creating listener address=:8881
stack_1 | DEBUG Loaded TLS certificate
stack_1 | DEBUG Creating listener address=:1882
stack_1 | DEBUG Creating listener address=:8882
stack_1 | DEBUG Loaded TLS certificate
stack_1 | DEBUG Creating listener address=:1881
stack_1 | DEBUG Creating listener address=:1887
stack_1 | DEBUG Creating listener address=:8887
stack_1 | DEBUG Loaded TLS certificate
stack_1 | DEBUG Creating listener address=:1883
stack_1 | DEBUG Creating listener address=:8883
stack_1 | DEBUG Loaded TLS certificate
stack_1 | 2020/09/25 09:04:37 http: TLS handshake error from MY_IP: remote error: tls: unknown certificate
stack_1 | INFO Request handled duration=27.119µs method=GET namespace=web remote_addr=MY_IP request_id=01EK283NWXE8DVR8AEKS38BM8B response_size=45 status=308 url=https://MY_HOST_NAME/console
stack_1 | INFO Request handled duration=2.179706ms method=GET namespace=web remote_addr=MY_IP request_id=01EK283NZMA2KJN2A0D0J03JRH response_size=790 status=200 url=https://MY_HOST_NAME/console/
stack_1 | INFO Request handled duration=294.377552ms method=GET namespace=web remote_addr=MY_IP request_id=01EK283P5YBHFZE53MS8FMJ9J5 response_size=273513 status=200 url=https://MY_HOST_NAME/assets/console.c6ddc5f8af3f8c39f150.css
stack_1 | INFO Client error duration=801.97µs method=GET namespace=web remote_addr=MY_IP request_id=01EK283QMNZH4FC96J6KN13901 response_size=198 status=401 url=https://MY_HOST_NAME/console/api/auth/token
stack_1 | INFO Request handled duration=231.216µs method=GET namespace=web remote_addr=MY_IP request_id=01EK283QY8F1E31DRJVR4K70JK response_size=5004 status=200 url=https://MY_HOST_NAME/assets/logo.svg
stack_1 | INFO Request handled duration=452.974µs method=GET namespace=web remote_addr=MY_IP request_id=01EK283QY87G62PMDG9TXKFG9N response_size=0 status=302 url=https://MY_HOST_NAME/console/login/ttn-stack?next=/
stack_1 | INFO Request handled duration=222.014µs method=GET namespace=web remote_addr=MY_IP request_id=01EK283R2GE450CC42B7V002ZP response_size=0 status=302 url=https://MY_HOST_NAME/oauth/authorize?client_id=console&redirect_uri=%2Fconsole%2Foauth%2Fcallback&response_type=code&state=fQ_ugtpulr20LYET
stack_1 | INFO Request handled duration=1.108407ms method=GET namespace=web remote_addr=MY_IP request_id=01EK283R63F33VA1CTVJT1XKAK response_size=681 status=200 url=https://MY_HOST_NAME/oauth/login?n=%2Foauth%2Fauthorize%3Fclient_id%3Dconsole%26redirect_uri%3D%252Fconsole%252Foauth%252Fcallback%26response_type%3Dcode%26state%3DfQ_ugtpulr20LYET
stack_1 | INFO Request handled duration=593.912µs method=GET namespace=web remote_addr=MY_IP request_id=01EK283RECZQ85S12T8XJSRGCC response_size=82865 status=200 url=https://MY_HOST_NAME/assets/oauth.7974b415b4e1ab2ca5a4.css
stack_1 | INFO Request handled duration=1.647218362s method=GET namespace=web remote_addr=MY_IP request_id=01EK283REDP6YFY9ZMWSVHR3H1 response_size=1318285 status=200 url=https://MY_HOST_NAME/assets/oauth.ac5a77c09e306783f4cc.js
stack_1 | INFO Client error duration=684.629µs method=GET namespace=web remote_addr=MY_IP request_id=01EK283TKSAD6A9P94ZV253WBV response_size=177 status=401 url=https://MY_HOST_NAME/oauth/api/me
stack_1 | INFO Request handled duration=529.266µs method=GET namespace=web remote_addr=MY_IP request_id=01EK283TXAH8577NWQD66DYP7Z response_size=25520 status=200 url=https://MY_HOST_NAME/assets/source-sans-pro-v13-latin_latin-ext-600.117e12cdb861ed7356c805f6f515afbb.woff2
stack_1 | INFO Request handled duration=105.461865ms method=POST namespace=web remote_addr=MY_IP request_id=01EK28425T333E7HBWC9MXDNVJ response_size=0 status=204 url=https://MY_HOST_NAME/oauth/api/auth/login
stack_1 | INFO Request handled duration=67.869363ms method=GET namespace=web remote_addr=MY_IP request_id=01EK2842CRCMA52JDRWSQ3PRXZ response_size=0 status=302 url=https://MY_HOST_NAME/oauth/authorize?client_id=console&redirect_uri=%2Fconsole%2Foauth%2Fcallback&response_type=code&state=fQ_ugtpulr20LYET
stack_1 | WARN error=unauthorized_client, internal_error= get_client=client check failed, client_id=console namespace=identityserver
stack_1 | WARN OAuth error error=error:pkg/oauth:unauthorized_client (client is not authorized to request a token using this method) method=POST namespace=web remote_addr=127.0.0.1 request_id=01EK2842HTM7Z187V2B6293699 url=http://localhost:1885/oauth/token
stack_1 | INFO Client error duration=29.157257ms method=POST namespace=web remote_addr=127.0.0.1 request_id=01EK2842HTM7Z187V2B6293699 response_size=209 status=403 url=http://localhost:1885/oauth/token
stack_1 | INFO Client error duration=32.610047ms method=GET namespace=web remote_addr=MY_IP request_id=01EK2842HRBM3EN2QDQVJCE1M1 response_size=996 status=403 url=https://MY_HOST_NAME/console/oauth/callback?code=A_LONG_CODE&state=fQ_ugtpulr20LYET
stack_1 | INFO Client error duration=866.96µs method=GET namespace=web remote_addr=MY_IP request_id=01EK28432GCDSC5232X7Q77CCY response_size=198 status=401 url=https://MY_HOST_NAME/console/api/auth/token
stack_1 | INFO Request handled duration=679.644831ms method=GET namespace=web remote_addr=MY_IP request_id=01EK28436M2P6NRDWR6SXR0JCZ response_size=699912 status=200 url=https://MY_HOST_NAME/assets/ttn-console-bg.498252edc489187693a4e32162260925.png

While copying this, I noticed somehing that might give us a clue:

http: TLS handshake error from MY_IP: remote error: tls: unknown certificate

Does any of this ring a bell to you?

Indeed I recall to have a similar issue and the error message was a bit misleading. Stupid question what os are you using and did you open the ports on the firewall ?

I’m running this on an Amazon Linux 2 AMI and I opened the ports 80, 443, 1884, 8884, 1885, 8885, 1887, 8887 which are the ports that are used for management and such as documented here https://thethingsstack.io/reference/networking/.

Mmm okay I had it running on a local dedicated server. Did you sign the certificate ?

Yes, I’m using self signed certificates. I tried doing the ACME approach at first but it was giving me an error saying that the certificates couldn’t be found.

Well in this case I’m sorry but I cannot help you further in the end :sweat_smile:

Something that I realized is that often the error messages provided are quite misleading. In my case it happened more than once that the issue was something totally different then the actual message was saying. Not a big help I know but yeah…

Good luck with your issue and keep us update if and how you managed to solve the issue

Update: I managed to make ACME work, so at least it will auto install the certificates where it needs to. I can now access via HTTPS correctly, but I still face the same Token Exchange Refused message.

Update 2: I DID IT! Turns out, you need to change most of the config options so that instead of localhost you point to your own domain name.

Example of a chunk of the changes I added to docker-compose.yml:
TTN_LW_CONSOLE_UI_AS_BASE_URL: https://mydomain:8885/api/v3
TTN_LW_CONSOLE_UI_IS_BASE_URL: https://mydomain:8885/api/v3
TTN_LW_OAUTH_SERVER_ADDRESS: https://mydomain:8885/oauth
TTN_LW_IS_OAUTH_UI_CANONICAL_URL: https://mydomain:8885/oauth
TTN_LW_IS_OAUTH_UI_IS_BASE_URL: https://mydomain:8885/api/v3
TTN_LW_APPLICATION_SERVER_GRPC_ADDRESS: mydomain:8884
TTN_LW_DEVICE_CLAIMING_SERVER_GRPC_ADDRESS: mydomain:8884

To know all the parameters you need to change just call ttn-lw-stack config | grep localhost and see what should be changed and what already works as is.

Also make sure the ports are open and accessible from the IP you are testing from AND from the server itself at least. Port 80 also needs to be open to Let’s Encrypt.

Also also I added the port 8885 to the command that creates the oauth client, so it ends up like this:
docker-compose run --rm stack is-db create-oauth-client --id console --name "Console" --owner admin --secret "console" --redirect-uri "https://mydomain:8885/console/oauth/callback" --redirect-uri "/console/oauth/callback" --logout-redirect-uri "https://mydomain:8885/console" --logout-redirect-uri "/console"
Not sure if this was really needed though, but it works for me so I’m not touching it ^^’.

2 Likes

Hi @christophmerscher if you don’t mind, I can use your help on how you got a self signed certificate working as part of your config? I am running a VM on a local network and using my own domain name? Eventually I will assign it to a public IP and use Let’s Encrypt automatically, but for now I would like some clear help on self signed certificate.

At the moment I have
Token Exchange Refused errors which as per configuration instructions is when you try to log in to The Things Stack Console, because an authorization request generated from within the container will not be redirected to port 1885 or 8885, where The Things Stack is listening.

Could you please help?

Hi @Tigere, I spent a bit of time facing this issue too but eventually got it to work. I was running TTS v3.9.4 in docker in a VM running Ubuntu 20.04, using self signed certificates in a made up domain name (ttn.local.com) that had not entries in any DNS.

To get mine to work I had to ensure the mapping of domain name to TTS stack docker network gateway ip address was put into both the VM machine /etc/hosts file AND in the TTS docker container /etc/hosts file (ie. 172.25.0.1 ttn.local.com).

Note to add the mapping to the tts stack docker container /etc/hosts file I found it easiest to add the extra-hosts property to the docker-compose.yml file so it adds the mapping to the docker /etc/hosts file automatically upon starting the container. Here’s a snippet of my docker-compose.yml file:
services:
…
stack:
…
extra_hosts:
- “ttn.local.com:172.25.0.1” # Ensure this IP address matches the docker network gateway address.
…

1 Like

Hi @TheIronNinja, the configuration options you’re specifying here are in our example ttn-lw-stack-docker.yml file -> https://thethingsstack.io/getting-started/installation/configuration/#example-configuration-files

If you set up The Things Stack according to the documentation, i.e with folder structure like

docker-compose.yml          # defines Docker services for running {{% tts %}}
config/
└── stack/
    └── ttn-lw-stack-docker.yml    # configuration file for {{% tts %}}

And you use our example docker-compose.yml file, when you start the stack, the entrypoint line loads the ttn-lw-stack-docker.yml file with all of the configuration options:

stack:
    # In production, replace 'latest' with tag from https://hub.docker.com/r/thethingsnetwork/lorawan-stack/tags
    image: thethingsnetwork/lorawan-stack:latest
    entrypoint: ttn-lw-stack -c /config/ttn-lw-stack-docker.yml

Glad you got this working. Running self signed certificates inside a Docker container with no DNS address is complicated! We’ve added instructions for using the machine’s local IP address, but adding the DNS records manually is a good solution.

@benolayinka, I can’t see any of @TheIronNinja’s config settings in the example ttn-lw-stack-docker.yml file. That file only contains TTN_LW_BLOB_LOCAL_DIRECTORY, TTN_LW_REDIS_ADDRESS, and TTN_LW_IS_DATABASE_URI. @TheIronNinja has specified 7 additional settings in their post, as well as adding port 8885 to the console oauth client registration.