Mistakenly changed the frequency plan and Region physical parameter and now I can not delete the end device

Hello,

I made a mistake while registering end device. After that I changed frequency plan and Region physical parameter from the end device settings in the console. (Later I found out that TTS does not support this). Now I can not access the settings of end device or neither delete the device or application.

What could be a solution here?

The command-line interface can often do things that the Console can’t. Have you tried that?

If not, can you first share the Application ID and Device ID, please? Then we can take a quick look to see why the Console doesn’t let you do this.

@htdvisser thanks for the quick response.

I have not tried with command line. I will look into it and try it.

I am running the open source version of the things stack via Docker on my private server. Still if sharing application ID and device ID will help, I can share them.

I am getting following error while access the said end device
image

However, I can access other end devices successfully.

This is a backend error. Details should be in Sentry (if you configured that for your deployment) or in the server logs. It would be helpful if you could copy-paste those so that we can investigate further.

I used docker logs --details -f command to get following logs

 runtime error: invalid memory address or nil pointer dereference
 goroutine 23864 [running]:
 runtime/debug.Stack()
        /opt/hostedtoolcache/go/1.17.5/x64/src/runtime/debug/stack.go:24 +0x65
 go.thethings.network/lorawan-stack/v3/pkg/rpcserver.New.func1({0x23025a0, 0x44c18c0})
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/rpcserver/rpcserver.go:146 +0x5a
 github.com/grpc-ecosystem/go-grpc-middleware/recovery.WithRecoveryHandler.func1.1({0xc00120ca50, 0x4ea025}, {0x23025a0, 0x44c18c0})
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/recovery/options.go:33 +0x2d
 github.com/grpc-ecosystem/go-grpc-middleware/recovery.recoverFrom({0x2c3d6c8, 0xc0013b3a10}, {0x23025a0, 0x44c18c0}, 0xc0000a21b0)
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/recovery/interceptors.go:61 +0x36
 github.com/grpc-ecosystem/go-grpc-middleware/recovery.UnaryServerInterceptor.func1.1()
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/recovery/interceptors.go:29 +0x7b
 panic({0x23025a0, 0x44c18c0})
        /opt/hostedtoolcache/go/1.17.5/x64/src/runtime/panic.go:1038 +0x215
 go.thethings.network/lorawan-stack/v3/pkg/crypto/cryptoutil.UnwrapSelectedSessionKeys({0x2c3d6c8, 0xc0013b3a10}, {0x2c87060, 0xc000615140}, 0x0, {0x26ee72a, 0xc001231680}, {0xc0001dc300, 0x5, 0x8})
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/crypto/cryptoutil/cryptoutil.go:126 +0x131
 go.thethings.network/lorawan-stack/v3/pkg/applicationserver.asEndDeviceRegistryServer.retrieveSessionKeys.func1({0x2c3d6c8, 0xc0013b3a10}, 0xc001377360, {0x26ee72a, 0xc00147bc20}, {0xc0001dc300, 0x2c3d6c8, 0xc0013b3a10})
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/applicationserver/grpc_deviceregistry.go:60 +0x71
 go.thethings.network/lorawan-stack/v3/pkg/applicationserver.asEndDeviceRegistryServer.retrieveSessionKeys({0xc000cfc500, {0x0, 0xc0013b3a10}}, {0x2c3d6c8, 0xc0013b3a10}, 0xc001342400, {0xc0001dc300, 0x5, 0x8})
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/applicationserver/grpc_deviceregistry.go:111 +0x42a
 go.thethings.network/lorawan-stack/v3/pkg/applicationserver.asEndDeviceRegistryServer.Get({0xc000cfc500, {0x0, 0x40adc5}}, {0x2c3d6c8, 0xc0013b3a10}, 0xc00094e588)
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/applicationserver/grpc_deviceregistry.go:157 +0x4e5
 go.thethings.network/lorawan-stack/v3/pkg/ttnpb._AsEndDeviceRegistry_Get_Handler.func1({0x2c3d6c8, 0xc0013b3a10}, {0x25e8aa0, 0xc00094e588})
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/ttnpb/applicationserver.pb.go:2347 +0x78
 go.thethings.network/lorawan-stack/v3/pkg/rpcmiddleware/hooks.UnaryServerInterceptor.func1({0x2c3d6c8, 0xc0013b3a10}, {0x25e8aa0, 0xc00094e588}, 0x21f9f60, 0xc00094e618)
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/rpcmiddleware/hooks/hooks.go:132 +0xe5
 github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1({0x2c3d6c8, 0xc0013b3a10}, {0x25e8aa0, 0xc00094e588})
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3a
 go.thethings.network/lorawan-stack/v3/pkg/ratelimit.UnaryServerInterceptor.func1({0x2c3d6c8, 0xc0013b3a10}, {0x25e8aa0, 0xc00094e588}, 0xc00116a060, 0xc00116a080)
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/ratelimit/grpc.go:73 +0x371
 github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1({0x2c3d6c8, 0xc0013b3a10}, {0x25e8aa0, 0xc00094e588})
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3a
 go.thethings.network/lorawan-stack/v3/pkg/rpcmiddleware/validator.UnaryServerInterceptor.func1({0x2c3d6c8, 0xc0013b3a10}, {0x25e8aa0, 0xc00094e588}, 0x114d4ef, 0xc00116a0e0)
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/rpcmiddleware/validator/validator.go:204 +0x84
 github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1({0x2c3d6c8, 0xc0013b3a10}, {0x25e8aa0, 0xc00094e588})
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3a
 github.com/grpc-ecosystem/go-grpc-middleware/recovery.UnaryServerInterceptor.func1({0x2c3d6c8, 0xc0013b3a10}, {0x25e8aa0, 0xc00094e588}, 0x2b425ce34dd97f40, 0x22dbb20)
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/recovery/interceptors.go:33 +0xc8
 github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1({0x2c3d6c8, 0xc0013b3a10}, {0x25e8aa0, 0xc00094e588})
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3a
 go.thethings.network/lorawan-stack/v3/pkg/rpcmiddleware/sentry.UnaryServerInterceptor.func1({0x2c3d6c8, 0xc0013b3a10}, {0x25e8aa0, 0xc00094e588}, 0xc00116a060, 0x40ac5d)
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/rpcmiddleware/sentry/sentry.go:141 +0x3d
 github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1({0x2c3d6c8, 0xc0013b3a10}, {0x25e8aa0, 0xc00094e588})
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3a
 go.thethings.network/lorawan-stack/v3/pkg/errors.UnaryServerInterceptor.func1({0x2c3d6c8, 0xc0013b3a10}, {0x25e8aa0, 0xc00094e588}, 0x7f8bc76dfc50, 0xc00130f590)
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/errors/grpc.go:116 +0x30
 github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1({0x2c3d6c8, 0xc0013b3a10}, {0x25e8aa0, 0xc00094e588})
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3a
 github.com/grpc-ecosystem/go-grpc-prometheus.(*ServerMetrics).UnaryServerInterceptor.func1({0x2c3d6c8, 0xc0013b3a10}, {0x25e8aa0, 0xc00094e588}, 0xc0011df678, 0xc00116a160)
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-prometheus@v1.2.0/server_metrics.go:107 +0x87
 github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1({0x2c3d6c8, 0xc0013b3a10}, {0x25e8aa0, 0xc00094e588})
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3a
 go.thethings.network/lorawan-stack/v3/pkg/rpcmiddleware/rpclog.UnaryServerInterceptor.func1({0x2c3d6c8, 0xc0013b3800}, {0x25e8aa0, 0xc00094e588}, 0xc00116a060, 0xc00116a180)
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/rpcmiddleware/rpclog/server_interceptors.go:37 +0x1c3
 github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1({0x2c3d6c8, 0xc0013b3800}, {0x25e8aa0, 0xc00094e588})
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3a
 go.thethings.network/lorawan-stack/v3/pkg/events.UnaryServerInterceptor({0x2c3d6c8, 0xc0013b37d0}, {0x25e8aa0, 0xc00094e588}, 0x0, 0xc00116a1a0)
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/events/correlation_interceptor.go:43 +0x59
 github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1({0x2c3d6c8, 0xc0013b37d0}, {0x25e8aa0, 0xc00094e588})
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3a
 github.com/grpc-ecosystem/go-grpc-middleware/tracing/opentracing.UnaryServerInterceptor.func1({0x2c3d6c8, 0xc0013b37a0}, {0x25e8aa0, 0xc00094e588}, 0x2301a00, 0xc00116a1c0)
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/tracing/opentracing/server_interceptors.go:38 +0x16a
 github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1({0x2c3d6c8, 0xc0013b37a0}, {0x25e8aa0, 0xc00094e588})
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3a
 go.thethings.network/lorawan-stack/v3/pkg/rpcmiddleware.(*ProxyHeaders).UnaryServerInterceptor.func1({0x2c3d6c8, 0xc0013b36e0}, {0x25e8aa0, 0xc00094e588}, 0xc000b8d880, 0xc00116a1e0)
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/rpcmiddleware/proxy_headers.go:87 +0x85
 github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1({0x2c3d6c8, 0xc0013b36e0}, {0x25e8aa0, 0xc00094e588})
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3a
 go.thethings.network/lorawan-stack/v3/pkg/rpcmiddleware.RequestIDUnaryServerInterceptor.func1({0x2c3d6c8, 0xc0013b36e0}, {0x25e8aa0, 0xc00094e588}, 0x0, 0xc00116a200)
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/rpcmiddleware/request_id.go:52 +0x59
 github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1({0x2c3d6c8, 0xc0013b36e0}, {0x25e8aa0, 0xc00094e588})
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3a
 github.com/grpc-ecosystem/go-grpc-middleware/tags.UnaryServerInterceptor.func1({0x2c3d6c8, 0xc0013b3680}, {0x25e8aa0, 0xc00094e588}, 0xc00116a060, 0xc00116a220)
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/tags/interceptors.go:23 +0xa6
 github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1({0x2c3d6c8, 0xc0013b3680}, {0x25e8aa0, 0xc00094e588})
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3a
 go.thethings.network/lorawan-stack/v3/pkg/rpcmiddleware/fillcontext.UnaryServerInterceptor.func1({0x2c3d6c8, 0xc0013b3590}, {0x25e8aa0, 0xc00094e588}, 0x18, 0xc00116a240)
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/rpcmiddleware/fillcontext/fillcontext.go:34 +0x5e
 github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1({0x2c3d6c8, 0xc0013b3590}, {0x25e8aa0, 0xc00094e588})
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25 +0x3a
 github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1({0x2c3d6c8, 0xc0013b3590}, {0x25e8aa0, 0xc00094e588}, 0xc000909bb8, 0x2301340)
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:34 +0xbf
 go.thethings.network/lorawan-stack/v3/pkg/ttnpb._AsEndDeviceRegistry_Get_Handler({0x24ef8e0, 0xc0009a5398}, {0x2c3d6c8, 0xc0013b3590}, 0xc0011fa1e0, 0xc000828510)
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/ttnpb/applicationserver.pb.go:2349 +0x138
 google.golang.org/grpc.(*Server).processUnaryRPC(0xc0001596c0, {0x2c88b10, 0xc000f26000}, 0xc00193f300, 0xc000b4dcb0, 0x44d9ea0, 0x0)
        /home/runner/go/pkg/mod/google.golang.org/grpc@v1.33.1/server.go:1210 +0xc8f
 google.golang.org/grpc.(*Server).handleStream(0xc0001596c0, {0x2c88b10, 0xc000f26000}, 0xc00193f300, 0x0)
        /home/runner/go/pkg/mod/google.golang.org/grpc@v1.33.1/server.go:1533 +0xa2a
 google.golang.org/grpc.(*Server).serveStreams.func1.2()
        /home/runner/go/pkg/mod/google.golang.org/grpc@v1.33.1/server.go:871 +0x98
 created by google.golang.org/grpc.(*Server).serveStreams.func1
        /home/runner/go/pkg/mod/google.golang.org/grpc@v1.33.1/server.go:869 +0x294
 ERROR  Finished unary call     {"auth.token_id": "DLPE5CLRBIJYYPDMMRWFFH5776EKKD65J3Y64KQ", "auth.token_type": "AccessToken", "auth.user_id": "admin", "duration": 1.3298, "error": "error:pkg/rpcserver:rpc_recovered (Internal Server Error)", "error_cause": "runtime error: invalid memory address or nil pointer dereference", "error_correlation_id": "a847b5143cab45808345d36189167ddf", "error_name": "rpc_recovered", "error_namespace": "pkg/rpcserver", "grpc.method": "Get", "grpc.request.application_id": "pce-wetter-station", "grpc.request.device_id": "eui-a84041df5182f086", "grpc.service": "ttn.lorawan.v3.AsEndDeviceRegistry", "grpc_code": "Internal", "namespace": "grpc", "peer.address": "pipe", "peer.real_ip": "78.94.20.122", "request_id": "01FZDGAFWMNN2W9ZQX86KM4PWJ"}
 go.thethings.network/lorawan-stack/v3/pkg/log.(*zapHandler).HandleLog
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/log/zap_handler.go:84
 go.thethings.network/lorawan-stack/v3/pkg/log.(*Logger).Use.func1
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/log/logger.go:55
 go.thethings.network/lorawan-stack/v3/pkg/log.HandlerFunc.HandleLog
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/log/handler.go:38
 go.thethings.network/lorawan-stack/v3/pkg/log/middleware/observability.(*observability).Wrap.func1
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/log/middleware/observability/observability.go:86
 go.thethings.network/lorawan-stack/v3/pkg/log.HandlerFunc.HandleLog
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/log/handler.go:38
 go.thethings.network/lorawan-stack/v3/pkg/log.(*Logger).commit
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/log/logger.go:75
 go.thethings.network/lorawan-stack/v3/pkg/log.(*entry).commit
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/log/entry.go:69
 go.thethings.network/lorawan-stack/v3/pkg/log.(*entry).Error
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/log/entry.go:94
 go.thethings.network/lorawan-stack/v3/pkg/rpcmiddleware/rpclog.commit
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/rpcmiddleware/rpclog/utils.go:103
 go.thethings.network/lorawan-stack/v3/pkg/rpcmiddleware/rpclog.UnaryServerInterceptor.func1
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/rpcmiddleware/rpclog/server_interceptors.go:70
 github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25
 go.thethings.network/lorawan-stack/v3/pkg/events.UnaryServerInterceptor
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/events/correlation_interceptor.go:43
 github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25
 github.com/grpc-ecosystem/go-grpc-middleware/tracing/opentracing.UnaryServerInterceptor.func1
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/tracing/opentracing/server_interceptors.go:38
 github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25
 go.thethings.network/lorawan-stack/v3/pkg/rpcmiddleware.(*ProxyHeaders).UnaryServerInterceptor.func1
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/rpcmiddleware/proxy_headers.go:87
 github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25
 go.thethings.network/lorawan-stack/v3/pkg/rpcmiddleware.RequestIDUnaryServerInterceptor.func1
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/rpcmiddleware/request_id.go:52
 github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25
 github.com/grpc-ecosystem/go-grpc-middleware/tags.UnaryServerInterceptor.func1
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/tags/interceptors.go:23
 github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25
 go.thethings.network/lorawan-stack/v3/pkg/rpcmiddleware/fillcontext.UnaryServerInterceptor.func1
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/rpcmiddleware/fillcontext/fillcontext.go:34
 github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1.1.1
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:25
 github.com/grpc-ecosystem/go-grpc-middleware.ChainUnaryServer.func1
        /home/runner/go/pkg/mod/github.com/grpc-ecosystem/go-grpc-middleware@v1.3.0/chain.go:34
 go.thethings.network/lorawan-stack/v3/pkg/ttnpb._AsEndDeviceRegistry_Get_Handler
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/ttnpb/applicationserver.pb.go:2349
 google.golang.org/grpc.(*Server).processUnaryRPC
        /home/runner/go/pkg/mod/google.golang.org/grpc@v1.33.1/server.go:1210
 google.golang.org/grpc.(*Server).handleStream
        /home/runner/go/pkg/mod/google.golang.org/grpc@v1.33.1/server.go:1533
 google.golang.org/grpc.(*Server).serveStreams.func1.2
        /home/runner/go/pkg/mod/google.golang.org/grpc@v1.33.1/server.go:871
 ERROR  Server error    {"auth.token_id": "DLPE5CLRBIJYYPDMMRWFFH5776EKKD65J3Y64KQ", "auth.token_type": "AccessToken", "duration": 1.4323, "http.method": "GET", "http.path": "/api/v3/as/applications/pce-wetter-station/devices/eui-a84041df5182f086", "http.status": 500, "namespace": "web", "peer.address": "78.94.20.122:57064", "peer.real_ip": "78.94.20.122", "request_id": "01FZDGAFWMNN2W9ZQX86KM4PWJ"}
 go.thethings.network/lorawan-stack/v3/pkg/log.(*zapHandler).HandleLog
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/log/zap_handler.go:84
 go.thethings.network/lorawan-stack/v3/pkg/log.(*Logger).Use.func1
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/log/logger.go:55
 go.thethings.network/lorawan-stack/v3/pkg/log.HandlerFunc.HandleLog
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/log/handler.go:38
 go.thethings.network/lorawan-stack/v3/pkg/log/middleware/observability.(*observability).Wrap.func1
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/log/middleware/observability/observability.go:86
 go.thethings.network/lorawan-stack/v3/pkg/log.HandlerFunc.HandleLog
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/log/handler.go:38
 go.thethings.network/lorawan-stack/v3/pkg/log.(*Logger).commit
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/log/logger.go:75
 go.thethings.network/lorawan-stack/v3/pkg/log.(*entry).commit
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/log/entry.go:69
 go.thethings.network/lorawan-stack/v3/pkg/log.(*entry).Error
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/log/entry.go:94
 go.thethings.network/lorawan-stack/v3/pkg/webmiddleware.Log.func1.1
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/webmiddleware/log.go:94
 net/http.HandlerFunc.ServeHTTP
        /opt/hostedtoolcache/go/1.17.5/x64/src/net/http/server.go:2047
 go.thethings.network/lorawan-stack/v3/pkg/webmiddleware.SecurityHeaders.func1.1
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/webmiddleware/security_headers.go:27
 net/http.HandlerFunc.ServeHTTP
        /opt/hostedtoolcache/go/1.17.5/x64/src/net/http/server.go:2047
 go.thethings.network/lorawan-stack/v3/pkg/webmiddleware.MaxBody.func1.1
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/webmiddleware/max_body.go:35
 net/http.HandlerFunc.ServeHTTP
        /opt/hostedtoolcache/go/1.17.5/x64/src/net/http/server.go:2047
 go.thethings.network/lorawan-stack/v3/pkg/webmiddleware.Metadata.func1.1
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/webmiddleware/metadata.go:43
 net/http.HandlerFunc.ServeHTTP
        /opt/hostedtoolcache/go/1.17.5/x64/src/net/http/server.go:2047
 go.thethings.network/lorawan-stack/v3/pkg/webmiddleware.ProxyHeaders.func1.1
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/webmiddleware/proxy_headers.go:108
 net/http.HandlerFunc.ServeHTTP
        /opt/hostedtoolcache/go/1.17.5/x64/src/net/http/server.go:2047
 go.thethings.network/lorawan-stack/v3/pkg/webmiddleware.RequestID.func1.1
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/webmiddleware/request_id.go:36
 net/http.HandlerFunc.ServeHTTP
        /opt/hostedtoolcache/go/1.17.5/x64/src/net/http/server.go:2047
 go.thethings.network/lorawan-stack/v3/pkg/webmiddleware.RequestURL.func1.1
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/webmiddleware/request_url.go:28
 net/http.HandlerFunc.ServeHTTP
        /opt/hostedtoolcache/go/1.17.5/x64/src/net/http/server.go:2047
 go.thethings.network/lorawan-stack/v3/pkg/webmiddleware.Peer.func1.1
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/webmiddleware/peer.go:34
 net/http.HandlerFunc.ServeHTTP
        /opt/hostedtoolcache/go/1.17.5/x64/src/net/http/server.go:2047
 go.thethings.network/lorawan-stack/v3/pkg/webmiddleware.FillContext.func1.1
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/webmiddleware/fillcontext.go:34
 net/http.HandlerFunc.ServeHTTP
        /opt/hostedtoolcache/go/1.17.5/x64/src/net/http/server.go:2047
 go.thethings.network/lorawan-stack/v3/pkg/webmiddleware.Recover.func1.1
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/webmiddleware/recover.go:49
 net/http.HandlerFunc.ServeHTTP
        /opt/hostedtoolcache/go/1.17.5/x64/src/net/http/server.go:2047
 go.thethings.network/lorawan-stack/v3/pkg/webhandlers.WithErrorHandlers.func1.1
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/webhandlers/error.go:74
 net/http.HandlerFunc.ServeHTTP
        /opt/hostedtoolcache/go/1.17.5/x64/src/net/http/server.go:2047
 github.com/gorilla/mux.(*Router).ServeHTTP
        /home/runner/go/pkg/mod/github.com/gorilla/mux@v1.8.0/mux.go:210
 go.thethings.network/lorawan-stack/v3/pkg/web.(*Server).ServeHTTP
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/web/web.go:309
 go.thethings.network/lorawan-stack/v3/pkg/component.(*Component).ServeHTTP
        /home/runner/work/lorawan-stack/lorawan-stack/pkg/component/component.go:405
 net/http.serverHandler.ServeHTTP
        /opt/hostedtoolcache/go/1.17.5/x64/src/net/http/server.go:2879
 net/http.initALPNRequest.ServeHTTP
        /opt/hostedtoolcache/go/1.17.5/x64/src/net/http/server.go:3480
 net/http.(*http2serverConn).runHandler
        /opt/hostedtoolcache/go/1.17.5/x64/src/net/http/h2_bundle.go:5849
 INFO   Finished unary call     {"auth.token_id": "DLPE5CLRBIJYYPDMMRWFFH5776EKKD65J3Y64KQ", "auth.token_type": "AccessToken", "auth.user_id": "admin", "duration": 1.4477, "grpc.method": "Get", "grpc.request.application_id": "pce-wetter-station", "grpc.request.device_id": "eui-a84041df5182f086", "grpc.service": "ttn.lorawan.v3.NsEndDeviceRegistry", "namespace": "grpc", "peer.address": "pipe", "peer.real_ip": "78.94.20.122", "request_id": "01FZDGAFWG2QCQYQA1RTWKAHYY"}
 INFO   Request handled {"auth.token_id": "DLPE5CLRBIJYYPDMMRWFFH5776EKKD65J3Y64KQ", "auth.token_type": "AccessToken", "duration": 1.4743, "http.method": "GET", "http.path": "/api/v3/ns/applications/pce-wetter-station/devices/eui-a84041df5182f086", "http.status": 200, "namespace": "web", "peer.address": "78.94.20.122:57064", "peer.real_ip": "78.94.20.122", "request_id": "01FZDGAFWG2QCQYQA1RTWKAHYY"}
 INFO   Finished unary call     {"auth.token_id": "DLPE5CLRBIJYYPDMMRWFFH5776EKKD65J3Y64KQ", "auth.token_type": "AccessToken", "auth.user_id": "admin", "duration": 1.6071, "error": "error:pkg/joinserver:device_not_found (device not found)", "error_correlation_id": "6715156f1a754c4ba3463c8c123e663a", "error_name": "device_not_found", "error_namespace": "pkg/joinserver", "grpc.method": "Get", "grpc.request.application_id": "pce-wetter-station", "grpc.request.device_id": "eui-a84041df5182f086", "grpc.service": "ttn.lorawan.v3.JsEndDeviceRegistry", "grpc_code": "NotFound", "namespace": "grpc", "peer.address": "pipe", "peer.real_ip": "78.94.20.122", "request_id": "01FZDGAFX747PXXDQWJYBYS5V3"}
 INFO   Client error    {"auth.token_id": "DLPE5CLRBIJYYPDMMRWFFH5776EKKD65J3Y64KQ", "auth.token_type": "AccessToken", "duration": 1.6168, "http.method": "GET", "http.path": "/api/v3/js/applications/pce-wetter-station/devices/eui-a84041df5182f086", "http.status": 404, "namespace": "web", "peer.address": "78.94.20.122:57064", "peer.real_ip": "78.94.20.122", "request_id": "01FZDGAFX747PXXDQWJYBYS5V3"}

Is this helpful? If there is another command or file to look into for logs please let me know. I am bit of novice.

This is helpful, thanks.

@adrianmares If I’m reading this right, we’re passing a nil *ttnpb.SessionKeys to UnwrapSelectedSessionKeys:

[...]/pkg/crypto/cryptoutil.UnwrapSelectedSessionKeys(
  {0x2c3d6c8, 0xc0013b3a10}, // ctx context.Context
  {0x2c87060, 0xc000615140}, // keyVault crypto.KeyVault
  0x0, // sk *ttnpb.SessionKeys
  {0x26ee72a, 0xc001231680}, // prefix string
  {0xc0001dc300, 0x5, 0x8}, // paths ...string
)

Perhaps caused by all: Remove embed and nullable from SessionKeys · TheThingsNetwork/lorawan-stack@180312e · GitHub? (deep link to file+line takes a while to load).

I’ll create an issue on GitHub for this.

@htdvisser Thanks. Could you also post the link of GitHub issue here so that I can follow it?

Which version of the stack are you running ? This particular trace should’ve been fixed in 3.18.0, per Fix miscellaneous panics by adriansmares · Pull Request #5172 · TheThingsNetwork/lorawan-stack · GitHub

Hmm, maybe that explains why I wasn’t able to reproduce this myself.

Should have asked for the version :man_facepalming:

@adrianmares Thank you for pointing it out. I am running 3.17. I will update to 3.18.

This topic was automatically closed 24 hours after the last reply. New replies are no longer allowed.