A Python program to listen to your devices with MQTT


(Rudolf Schimmel) #1

A python program to listen to your devices with mqttn with credentials

I will upload a Spark version when ready.

# node.py
# Simple Python client to show node activity from ttn MQTT brooker with credentials
# Author: R.Schimmel
# www.schimmel-bisolutions.nl
# first install paho.mqtt.client: pip install paho-mqtt
# 
import paho.mqtt.client as mqtt

#Call back functions 

# gives connection message
def on_connect(client,userdata,rc):
    print("Connected with result code:"+str(rc))
    # subscribe for all devices of user
    client.subscribe('+/devices/+/up')

# gives message from device
def on_message(client,userdata,msg):
    print"Topic",msg.topic + "\nMessage:" + str(msg.payload)

def on_log(client,userdata,level,buf):
    print("message:" + str(buf))
    print("userdata:" + str(userdata))
    
mqttc= mqtt.Client()
mqttc.on_connect=on_connect
mqttc.on_message=on_message

mqttc.username_pw_set("App-eu registered op ttn dashboard","key registered op ttn dashboard")

# BEWARE, outdated; see the follow up posts to use eu.thethings.network instead
mqttc.connect("staging.thethingsnetwork.org",1883,10)

# and listen to server
run = True
while run:
    mqttc.loop()

Is it possible to be connected to TTN with mqttws31.js
TTN microclient
(Mr The Wheel) #2

Ruud,

I'm a LoRa newbie so bear with me please.

I have a LoRa node (Sodaq Explorer) that send out data every now and than. I can see this data comming in at the MQTT server with this command:

mosquitto_sub -h eu.thethings.network -t '+/devices/+/up' -u '<Application ID>' -P '<Access Keys>' -v

Gives the messages:

<Application ID>/devices/<Device ID>/up {"app_id":"<Application ID>","dev_id":"<Device ID>","hardware_serial":"0004A30B001BABF7","port":1,"counter":877,"payload_raw":"SGVsbG8gWzEwOV0A","metadata":{"time":"2017-08-19T13:52:16.030688605Z","frequency":868.1,"modulation":"LORA","data_rate":"SF7BW125","coding_rate":"4/5","gateways":[{"gtw_id":"eui-b827ebffffe8ddf2","timestamp":2376170187,"time":"","channel":0,"rssi":-48,"snr":9,"latitude":52.34849,"longitude":5.21332,"altitude":-5}],"latitude":52.34853,"longitude":5.2133574,"altitude":-2,"location_source":"registry"}}

So, adopted your python program in the hope to be able to read and decode the messages from my node .. but it does not!

I changed the mqttc.username_pw_set() to:

mqttc.username_pw_set("<Application ID>","<Acces Keys>")

and:

mqttc.connect("eu.thethings.network",1883,60)

Running the program states:

Connected with result code:5

Resetting the connect statement to "staging.thethingsnetwork.org" gets rid of the error code 5 but nothings shown on the output...

I obvious are missing something.
Help would be very much apriciated!


(Arjan) #3

From the library's documentation:

The value of rc indicates success or not:

0: Connection successful 1: Connection refused - incorrect protocol version 2: Connection refused - invalid client identifier 3: Connection refused - server unavailable 4: Connection refused - bad username or password 5: Connection refused - not authorised 6-255: Currently unused.

Are you sure you used the application id (the one you entered yourself, and like you also used in mosquitto_sub), not the AppEUI (the one that TTN generates)?

(It's not a secret; anyone can see that id. The Python example works fine for me, though with a more recent version of the library one needs 4 parameters in def on_connect(client, userdata, flags, rc): )


(Mr The Wheel) #4

Thanks for your reply!

If I enter everything as I do with the bare mosquitto_sub command, I get the code 5 error.

If I use the mqtt broker as in the example, I do nót get an error but nothing shows op in the output (although the mosquitto_sub in an other window does show messages passing by).

According to the examples I found the user name should be the AppEUI..
I have made an application with the name "loraexplora" and the device has the same name.
The password of the application is generated by TTN and shows at the AppKeys.

I'm really confused by the functions of all these keys and ID's :neutral_face:

But, as far as I do understand Mosquitto, what I type in the cli version should be the same as what I type in the python script .. but that does not seem the case here....

Hope you can enlight me some more.


(Arjan) #5

No; see https://www.thethingsnetwork.org/docs/applications/mqtt/quick-start.html#credentials (You might be looking at old examples, which refer to the obsolete staging.thethingsnetwork.org.)

The access key you’re using should at least have access to the messages. You could generate an additional key just for your MQTT client.

Indeed, and so it does for me. I’d expect that too. (I have not tested with mosquitto_sub today.) The user name is the Application ID you’ve chosen yourself and the key starts with ttn-account-v2:

mqttc.username_pw_set("loraexplora", "ttn-account-v2.Cz...<redacted>...")
mqttc.connect("eu.thethings.network", 1883, 10)

Maybe update the Python MQTT library: pip install paho-mqtt --upgrade (and add the 4th parameter to on_connect like shown earlier.)


(Mr The Wheel) #6

Ok, thanks again for all the help!

It seems TTN authentification does not like an Application and Device with the same name :frowning:

So I added an other application with the name "my_second_lora_app" and connected the original "loraexplora" device at it.

Than, with this program I can connect to the MQTT server and decode the send messages:

#!/<path-to>/python
# Get date from MQTT server

import paho.mqtt.client as mqtt
import json
import base64

APPEUI = "70B3D57EF00069A3"
APPID  = "my_second_lora_app"
PSW    = 'ttn-account-v2.<lots-of-chars>'

#Call back functions 

# gives connection message
def on_connect(mqttc, mosq, obj,rc):
    print("Connected with result code:"+str(rc))
    # subscribe for all devices of user
    mqttc.subscribe('+/devices/#')

# gives message from device
def on_message(mqttc,obj,msg):
    x = json.loads(msg.payload)
    device = x["dev_id"]
    payload_raw = x["payload_raw"]
    payload_plain = base64.b64decode(payload_raw)
    datetime = x["metadata"]["time"]
    #rssi = x["metadata"]["gateways"]["rssi"]  # <== this raises an error (why?)
    rssi = -1
    print(device + ": " + payload_raw + " ==> " + payload_plain + ", RSSI ["+ str(rssi) + "] @" + datetime )

def on_publish(mosq, obj, mid):
    print("mid: " + str(mid))

def on_subscribe(mosq, obj, mid, granted_qos):
    print("Subscribed: " + str(mid) + " " + str(granted_qos))

def on_log(mqttc,obj,level,buf):
    print("message:" + str(buf))
    print("userdata:" + str(obj))
    
mqttc= mqtt.Client()
# Assign event callbacks
mqttc.on_connect=on_connect
mqttc.on_message=on_message

mqttc.username_pw_set(APPID, PSW)
mqttc.connect("eu.thethings.network",1883,60)

# and listen to server
run = True
while run:
    mqttc.loop()

(Gpolder) #7

because gateways is an array,
this: rssi = x[“metadata”][“gateways”][0][“rssi”]

works fine.


(Irnasslovenia) #8

I have improved the code a bit to work in Python3 and provide a simple CSV output showing packages received and signal strengths for each gateway.

Output example: > 2018-04-15T13:52:31.521984942Z, drone-mapper-new, 319, eui-3535303229005f00, -37, {‘alt’: 252, ‘lat’: 46.565055642429314, ‘lon’: 15.658404568338653, ‘hdop’: 2.2}

# https://www.thethingsnetwork.org/forum/t/a-python-program-to-listen-to-your-devices-with-mqtt/9036/6
# Get data from MQTT server
# Run this with python 3, install paho.mqtt prior to use

import paho.mqtt.client as mqtt
import json
import base64

APPEUI = "70B3D57EF00069A3"
APPID  = "my_second_lora_app"
PSW    = 'ttn-account-v2.<lots-of-chars>'

#Call back functions

# gives connection message
def on_connect(mqttc, mosq, obj,rc):
    print("Connected with result code:"+str(rc))
    # subscribe for all devices of user
    mqttc.subscribe('+/devices/+/up')

# gives message from device
def on_message(mqttc,obj,msg):
    try:
        #print(msg.payload)
        x = json.loads(msg.payload.decode('utf-8'))
        device = x["dev_id"]
        counter = x["counter"]
        payload_raw = x["payload_raw"]
        payload_fields = x["payload_fields"]
        datetime = x["metadata"]["time"]
        gateways = x["metadata"]["gateways"]
        # print for every gateway that has received the message and extract RSSI
        for gw in gateways:
            gateway_id = gw["gtw_id"]
            rssi = gw["rssi"]
            print(datetime + ", " + device + ", " + str(counter) + ", "+ gateway_id + ", "+ str(rssi) + ", " + str(payload_fields))
    except Exception as e:
        print(e)
        pass

def on_publish(mosq, obj, mid):
    print("mid: " + str(mid))

def on_subscribe(mosq, obj, mid, granted_qos):
    print("Subscribed: " + str(mid) + " " + str(granted_qos))

def on_log(mqttc,obj,level,buf):
    print("message:" + str(buf))
    print("userdata:" + str(obj))

mqttc= mqtt.Client()
# Assign event callbacks
mqttc.on_connect=on_connect
mqttc.on_message=on_message

mqttc.username_pw_set(APPID, PSW)
mqttc.connect("eu.thethings.network",1883,60)

# and listen to server
run = True
while run:
    mqttc.loop()