Registering a sensor in an TTN application via API

[LoRa_AuH]

Good morning everyone,

we have a web application and we want to build a device registration page to register a sensor in TTN V3 from our web application. Does anybody recommend me the right API to do this? Thank you so much.

Kind regards
Imran Khan

[descartes (Nick McCloud)]

Read the docs but avoid the Web API.

Once you’ve read the docs, then ask questions.

Alternatively, commission someone to perform an evaluation based on your use case.

And don’t forget, the final responsibility is yours.

[LoRa_AuH]

@descartes

Once you’ve read the docs, then ask questions.

Hello descartes,

I have an EndDDeviceRegistry Post request (https://eu1.cloud.thethings.network/api/v3/applications/dragino-soil-lse01/devices) in Postman.
This is the body of my Postman request. I want to just create a new device with the given name.

This is the response Postman response. I think the API is unable to post the locations data. I already have tried to give the longitude and latitude in the locations but I got the same error. Can you please guide me what is wrong with the Post Request? Thank you so much.

[rikietje]

Hey!

I was struggling with the same thing as you were a while ago.
As per documentation you need to register it to 4 servers.

This is how I managed it in Python (django):
Getting the values from my database object, created by the user, to be registered in TTN:

@receiver(post_save, sender=Device)
def register_device_into_ttn(sender, instance, created, **kwargs):
    if created:
        last_name_appendix = [last_name for last_name in instance.user.last_name.split(' ')]
        device_id_appendix = ('-' + instance.user.first_name + '-' + last_name_appendix[-1])[:16]
        device_id = (instance.device_id + device_id_appendix).casefold()
        app_key = instance.app_key.capitalize()
        dev_eui = instance.dev_eui.capitalize()
        app_eui = instance.app_eui.capitalize()
        app_id = instance.device_type.app_id

        data1 = {"end_device": {
            "ids": {"device_id": device_id, "dev_eui": dev_eui, "join_eui": app_eui},
            "join_server_address": "eu1.cloud.thethings.network",
            "network_server_address": "eu1.cloud.thethings.network",
            "application_server_address": "eu1.cloud.thethings.network", "name": device_id}, "field_mask": {
            "paths": ["join_server_address", "network_server_address", "application_server_address",     "ids.dev_eui",
                  "ids.join_eui", "name"]}}
        data2 = {"end_device": {
            "ids": {"device_id": device_id, "dev_eui": dev_eui, "join_eui": app_eui},
            "network_server_address": "eu1.cloud.thethings.network",
            "application_server_address": "eu1.cloud.thethings.network", "network_server_kek_label": "",
            "application_server_kek_label": "", "application_server_id": "", "net_id": None,
            "root_keys": {"app_key": {"key": app_key}}}, "field_mask": {
            "paths": ["network_server_address", "application_server_address", "ids.device_id", "ids.dev_eui",
                  "ids.join_eui", "network_server_kek_label", "application_server_kek_label",
                  "application_server_id",
                  "net_id", "root_keys.app_key.key"]}}

        data3 = {"end_device": {"multicast": False, "supports_join": True, "lorawan_version":     "MAC_V1_0_2",
                            "ids": {"device_id": device_id, "dev_eui": dev_eui,
                             "join_eui": app_eui}, "mac_settings": {"supports_32_bit_f_cnt": True},
                            "supports_class_c": False, "supports_class_b": False,
                            "lorawan_phy_version": "PHY_V1_0_2_REV_B", "frequency_plan_id": "EU_863_870_TTN"},
                 "field_mask": {
                     "paths": ["multicast", "supports_join", "lorawan_version", "ids.device_id", "ids.dev_eui",
                           "ids.join_eui", "mac_settings.supports_32_bit_f_cnt", "supports_class_c",
                           "supports_class_b", "lorawan_phy_version", "frequency_plan_id"]}}

        data4 = {"end_device": {
            "ids": {"device_id": device_id, "dev_eui": dev_eui, "join_eui": app_eui}},
            "field_mask": {"paths": ["ids.device_id", "ids.dev_eui", "ids.join_eui"]}}

        url1 = 'https://eu1.cloud.thethings.network/api/v3/applications/' + app_id + '/devices'
        url2 = 'https://eu1.cloud.thethings.network/api/v3/js/applications/' + app_id + '/devices'
        url3 = 'https://eu1.cloud.thethings.network/api/v3/ns/applications/' + app_id + '/devices'
        url4 = 'https://eu1.cloud.thethings.network/api/v3/as/applications/' + app_id + '/devices'

        headers = {
                "Authorization": "Bearer xxxx",
                'Content-Type': 'application/json'}

        responses = [requests.post(url1, data=json.dumps(data1), headers=headers),
                     requests.post(url2, data=json.dumps(data2), headers=headers),
                     requests.post(url3, data=json.dumps(data3), headers=headers),
                     requests.post(url4, data=json.dumps(data4), headers=headers)]

Hope this can help you out for the idea!

[LoRa_AuH]

@rikietje

Hope this can help you out for the idea!

Thank you so much rikietje. Thats work.

[rikietje]

You’re welcome!
A very good tip @descartes gave me when I was struggling with this a while ago was to use network page in the browser inspect while registering or deleting devices.

It makes the same API calls and you can see the formats etc.

Good luck!

[LoRa_AuH]

Hello rikietje,

I have implemented my Post Request in PHP the same as you did in Python. The device registry and network server responses are fine but I am getting errors in joint and application servers post requests.

Response of data1

 {
     “ids”:{
         “device_id”:“boden-07”,
         “application_ids”:{
                          “application_id”:“boden-messung”
                             },
         “dev_eui”:“647FDA0000006FC8”,
         “join_eui”:“647FDA8010000100”
            },
         “created_at”:“2021-05-31T14:35:18.966Z”,
         “updated_at”:“2021-05-31T14:35:18.966Z”,
         “name”:“boden-07”,
         “version_ids”{},
         “network_server_address”:“eu1.cloud.thethings.network”,
         “application_server_address”:“eu1.cloud.thethings.network”,
         “join_server_address”:“eu1.cloud.thethings.network”
            }

Response of data2

{
“code”:3,“message”:“error:pkg/rpcmiddleware/validator:field_mask_paths (forbidden path(s) in field mask)”,
“details” [{
“@type”:“type.googleapis.com/ttn.lorawan.v3.ErrorDetails",
“namespace”:“pkg/rpcmiddleware/validator”,
“name”:“field_mask_paths”,
“message_format”:"forbidden
path(s) in field mask”,
“attributes”:{
“forbidden_paths”:[
“boden-07”,
“647FDA0000006FC8”,
“647FDA8010000100”,
“”,
“”,
“”,
“”,
“F8A9C76B75013D14873807577430FF73”
]
},
“correlation_id”:“95c4a05d3f2840b4be1827d6f0fe2c71”,
“code”:3
}
]
}

Response of data3

 {
    “ids”:{
          “device_id”:“boden-07”,
          “application_ids”:{
                “application_id”:“boden-messung”
                },
          “dev_eui”:“647FDA0000006FC8”,
          “join_eui”:“647FDA8010000100”
                },
          “created_at”:“2021-05-31T14:35:19.477465462Z”,
          “updated_at”:“2021-05-31T14:35:19.477465462Z”
                }

Response of data4

 {
        “code”:3,
        “message”:“error:pkg/rpcmiddleware/validator:field_mask_paths (forbidden path(s) in field mask)”,
        “details”:[{
        "@type":“[type.googleapis.com/ttn.lorawan.v3.ErrorDetails",
        “namespace”:“pkg/rpcmiddleware/validator”,
        “name”:“field_mask_paths”,
        “message_format”:"forbidden
        ](http://type.googleapis.com/ttn.lorawan.v3.ErrorDetails%22,
        %22namespace%22:%22pkg/rpcmiddleware/validator%22,
        %22name%22:%22field_mask_paths%22,
        %22message_format%22:%22forbidden) path(s) in field mask”,
        “attributes”:{
        “forbidden_paths”:[
        “boden-07”,
        “647FDA0000006FC8”,
        “647FDA8010000100”
        ]
        },
        “correlation_id”:“2c93bda3a9654c5e987c5fff069d9007”,
        “code”:3
        }
        ]
        }

This is the response to my all four post requests.

$url1 = 'https://eu1.cloud.thethings.network/api/v3/applications/'.$this->appId.'/devices';
$url2 = 'https://eu1.cloud.thethings.network/api/v3/js/applications/'.$this->appId.'/devices';
$url3 = 'https://eu1.cloud.thethings.network/api/v3/ns/applications/'.$this->appId.'/devices';
$url4 = 'https://eu1.cloud.thethings.network/api/v3/as/applications/'.$this->appId.'/devices';

The above are the four URLs.
The four JSON data (data1, data2, data3, data4) are the same as you implemented.
Can you please guide me a bit on why I am getting this forbidden field_mask_paths error. Thank you so much.

Kind regards
Imran Khan

[LoRa_AuH]

@descartes

Read the docs but avoid the Web API.

Hello descartes,
can you please check my error and give me a valuable suggestion? Thanks

[descartes (Nick McCloud)]

Yes, but only if you repost that black & green blob as text. If it can be copied & pasted, it should be.

[descartes (Nick McCloud)]

Close, but what was wrong with the formatting options </>

How do I format my forum post? [HowTo]

The TTN forum is using Discourse, which supports several ways to format your post: Markdown BBCode HTML Any combination of the above Quite often you’ll need an empty line above specific formatting to make it work. You’ll also have to type blank lines to create proper paragraphs; just pressing Return once and then typing a new block of text right below a previous block only creates a so-called line break, which might be harder to read. Leaving one blank line between two blocks of texts nicely…

[rikietje]

Are you using post to register new devices? I got the same error when I was using put

[LoRa_AuH]

Yes I am using Post to register my device.

[LoRa_AuH]

Post request response of $data1 is to register the device:

{
    “ids”:{
        “device_id”:“boden-07”,
        “application_ids”:{
                          “application_id”:“boden-messung”
                           },
    “dev_eui”:“647FDA0000006FC8”,
    “join_eui”:“647FDA8010000100”
    },
    “created_at”:“2021-05-31T14:35:18.966Z”,
    “updated_at”:“2021-05-31T14:35:18.966Z”,
    “name”:“boden-07”,
    “version_ids”{},
    “network_server_address”:“eu1.cloud.thethings.network”,
    “application_server_address”:“eu1.cloud.thethings.network”,
    “join_server_address”:“eu1.cloud.thethings.network”
    }

Post request response of $data2 is for the join server:

    {
    “code”:3,“message”:“error:pkg/rpcmiddleware/validator:field_mask_paths (forbidden path(s) in field mask)”,
    “details” [{
    "@type":“[type.googleapis.com/ttn.lorawan.v3.ErrorDetails",
    “namespace”:“pkg/rpcmiddleware/validator”,
    “name”:“field_mask_paths”,
    “message_format”:"forbidden
    ](http://type.googleapis.com/ttn.lorawan.v3.ErrorDetails%22,%22namespace%22:%22pkg/rpcmiddleware/validator%22,%22name%22:%22field_mask_paths%22,%22message_format%22:%22forbidden) path(s) in field mask”,
    “attributes”:{
    “forbidden_paths”:[
    “boden-07”,
    “647FDA0000006FC8”,
    “647FDA8010000100”,
    "",
    "",
    "",
    "",
    “F8A9C76B75013D14873807577430FF73”
    ]
    },
    “correlation_id”:“95c4a05d3f2840b4be1827d6f0fe2c71”,
    “code”:3
    }
    ]
    }

Post request response of $data3 is for the network server:

{
“ids”:{
“device_id”:“boden-07”,
“application_ids”:{
“application_id”:“boden-messung”
},
“dev_eui”:“647FDA0000006FC8”,
“join_eui”:“647FDA8010000100”
},
“created_at”:“2021-05-31T14:35:19.477465462Z”,
“updated_at”:“2021-05-31T14:35:19.477465462Z”
}

Post request response of $data4 is for the application server:

{
    “code”:3,
    “message”:“error:pkg/rpcmiddleware/validator:field_mask_paths (forbidden path(s) in field mask)”,
    “details”:[{
    "@type":“[type.googleapis.com/ttn.lorawan.v3.ErrorDetails",
    “namespace”:“pkg/rpcmiddleware/validator”,
    “name”:“field_mask_paths”,
    “message_format”:"forbidden
    ](http://type.googleapis.com/ttn.lorawan.v3.ErrorDetails%22,
    %22namespace%22:%22pkg/rpcmiddleware/validator%22,
    %22name%22:%22field_mask_paths%22,
    %22message_format%22:%22forbidden) path(s) in field mask”,
    “attributes”:{
    “forbidden_paths”:[
    “boden-07”,
    “647FDA0000006FC8”,
    “647FDA8010000100”
    ]
    },
    “correlation_id”:“2c93bda3a9654c5e987c5fff069d9007”,
    “code”:3
    }
    ]
    }

These are all four responses to my four Post requests. I hope now it will be more readable and clear to you. Thank you so much.

[descartes (Nick McCloud)]

Really? I mean really?

Just edit your original post and format the log sections using the </> tool on the editor box please. The bold headings are nice, but it’s the computer data we ask people to format.

And then read the how to format your post topic please.

[LoRa_AuH]

@descartes

Really? I mean really?

Sorry, it was my mistake. Now I tried my best to format the post according to your expectations. Thanks.

[descartes (Nick McCloud)]

My expectations are the same as everyone else’s and were written down in the post on how to format posts with the link I gave you and I told you about the </> - it really helps if people read answers. We are volunteers here, we aren’t paid to wade through dense blocks of text that don’t provide structure.

Anyway, in the meanwhile, the Python works (as expected) so there will be something about your PHP conversion. I’m somewhat scared to ask you to post your code, but, using the power you have gained with the </> tool, can you show us your PHP please.

[LoRa_AuH]

@descartes

can you show us your PHP please

PHP does not understand JSON format so what I did is, I packed everything in array and converted it into json. The arrays of all four data are given below. In the field_mask field, I inserted directly the value of the variables.

                $data1 = [
                       'end_device' => [
                                   'ids' => [
                                       'device_id' => $this->deviceId,
                                        'dev_eui' => $this->devEui,
                                        'join_eui' => $this->appEui
                                        ],
                            'join_server_address' => 'eu1.cloud.thethings.network',
                            'network_server_address' => 'eu1.cloud.thethings.network',
                            'application_server_address' => 'eu1.cloud.thethings.network',
                            'name' => $this->deviceId
                                ],
                         'field_mask' => [
                                 'paths' => [
                                        'eu1.cloud.thethings.network',
                                        'eu1.cloud.thethings.network',
                                        'eu1.cloud.thethings.network',
                                        $this->devEui,
                                        $this->appEui,
                                        $this->deviceId
                                        ],
                                 ],
                            ];


              $data2 = [
                            'end_device' => [
                                        'ids' => [
                                                'device_id' => $this->deviceId,
                                                'dev_eui' => $this->devEui,
                                                'join_eui' => $this->appEui
                                                ],
                                'network_server_address' => 'eu1.cloud.thethings.network',
                                'application_server_address' => 'eu1.cloud.thethings.network',
                                'network_server_kek_label' => '',
                                'application_server_kek_label' => '',
                                'application_server_id' => '',
                                'net_id' => Null,
                                'root_keys' => [
                                          'app_key' => [
                                                  'key' => $this->appKey
                                        ],
                                    ],
                                    ],
                                'field_mask' => [
                                        'paths' => [
                                                 $this->deviceId,
                                                 $this->devEui,
                                                 $this->appEui,
                                                '',
                                                '',
                                                '',
                                                Null,
                                                $this->appKey
                                             ],
                                        ],
                                ];



                 $data3 = [
                        'end_device' => [
                                    'multicate' => False,
                                    'supports_join' => True,
                                    'lorawan_version' => $this->loraVersion,
                                    'ids' => [
                                          'device_id' => $this->deviceId,
                                           'dev_eui' => $this->devEui,
                                           'join_eui' => $this->appEui
                                        ],
                        'mac_settings' => [
                                        'supports_32_bit_f_cnt' => True
                                ],
                        'supports_class_c' => False,
                        'supports_class_b' => False,
                        'lorawan_phy_version' => $this->phyVersion,
                        'frequency_plan_id' => $this->frequencyPlan,
                         ],
                         'field_mask' => [
                                  'path' => [
                                         False,
                                         True,
                                         $this->loraVersion,
                                         $this->deviceId,
                                         $this->devEui,
                                         $this->appEui,
                                         True,
                                         False,
                                         False,
                                         $this->phyVersion,
                                         $this->frequencyPlan
                                        ],
                                    ],
                            ];

      

             $data4 = [
                      'end_device' => [
                                'ids' => [
                                    'device_id' => $this->deviceId,
                                    'dev_eui' => $this->devEui,
                                    'join_eui' => $this->appEui
                                ],
                                ],
                        'field_mask' => [
                                 'paths' => [
                                        $this->deviceId,
                                        $this->devEui,
                                        $this->appEui
                                    ],
                            ],
                        ];

I converted all of them into JSON using json_encode()

$data_end_device= json_encode($data1);
$data_join_server = json_encode($data2);
$data_network_server = json_encode($data3);
$data_application_server = json_encode($data4);

I am using bearer authentication to make the PHP Post request.

$url1 = 'https://eu1.cloud.thethings.network/api/v3/applications/'.$this->appId.'/devices';
$url2 = 'https://eu1.cloud.thethings.network/api/v3/js/applications/'.$this->appId.'/devices';
$url3 = 'https://eu1.cloud.thethings.network/api/v3/ns/applications/'.$this->appId.'/devices';
$url4 = 'https://eu1.cloud.thethings.network/api/v3/as/applications/'.$this->appId.'/devices';

 $app_api = Application::where('app_id', $this->appId)->first()->api_key;
 
$authorization = "Authorization: Bearer " . $app_api;

$ch1 = curl_init($url1);
curl_setopt($ch1, CURLOPT_HTTPHEADER, array($authorization, 'Content-Type: application/json'));
curl_setopt($ch1, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch1, CURLOPT_POSTFIELDS, $data_end_device);
curl_setopt($ch1, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch1, CURLOPT_FOLLOWLOCATION, 1);
$result1 = curl_exec($ch1);
curl_close($ch1);
                

$ch2 = curl_init($url2);
curl_setopt($ch2, CURLOPT_HTTPHEADER, array($authorization, 'Content-Type: application/json'));
curl_setopt($ch2, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch2, CURLOPT_POSTFIELDS, $data_join_server);
curl_setopt($ch2, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch2, CURLOPT_FOLLOWLOCATION, 1);
$result2 = curl_exec($ch2);
curl_close($ch2);

                
$ch3 = curl_init($url3);
curl_setopt($ch3, CURLOPT_HTTPHEADER, array($authorization, 'Content-Type: application/json'));
curl_setopt($ch3, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch3, CURLOPT_POSTFIELDS, $data_network_server);
curl_setopt($ch3, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch3, CURLOPT_FOLLOWLOCATION, 1);
$result3 = curl_exec($ch3);
curl_close($ch3);

$ch4 = curl_init($url4);
curl_setopt($ch4, CURLOPT_HTTPHEADER, array($authorization, 'Content-Type: application/json'));
curl_setopt($ch4, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch4, CURLOPT_POSTFIELDS, $data_application_server);
curl_setopt($ch4, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch4, CURLOPT_FOLLOWLOCATION, 1);
$result4 = curl_exec($ch4);
curl_close($ch4);

[descartes (Nick McCloud)]

Ta da!!!

OK, will have a run at this later this morning after I’ve unbroken my iMac after a bad OS install test yesterday.

[LoRa_AuH]

@descartes

OK, will have a run at this later this morning after I’ve unbroken my iMac after a bad OS install test yesterday.

Many thanks

[LoRa_AuH]

@descartes

OK, will have a run at this later this morning after I’ve unbroken my iMac after a bad OS install test yesterday.

Good morning descartes,
I have solved my last issue. Now all of my four Post requests went fine and got positive responses. I actually provided the wrong parameters in the field_mask field. Instead of providing values in the field_mask, I provided the fields that should be used and it worked fine. As an example, I am providing one of my data below.

               $data2 = [
                    'end_device' => [
                                'ids' => [
                                        'device_id' => $this->deviceId,
                                        'dev_eui' => $this->devEui,
                                        'join_eui' => $this->appEui
                                        ],
                        'network_server_address' => 'eu1.cloud.thethings.network',
                        'application_server_address' => 'eu1.cloud.thethings.network',
                        'network_server_kek_label' => '',
                        'application_server_kek_label' => '',
                        'application_server_id' => '',
                        'net_id' => Null,
                        'root_keys' => [
                                  'app_key' => [
                                          'key' => $this->appKey
                                ],
                            ],
                            ],
                        'field_mask' => [
                                'paths' => [
                                    'network_server_address', 
                                    'application_server_address', 
                                    'ids.device_id', 
                                    'ids.dev_eui',
                                    'ids.join_eui', 
                                    'network_server_kek_label', 
                                    'application_server_kek_label',
                                    'application_server_id',
                                    'net_id', 
                                    'root_keys.app_key.key'
                                     ],
                                ],
                        ];

Below is the response of my first Post request.

{"
 ids":{
 "device_id":"boden-07",
 "application_ids":{
 "application_id":"boden-messung"
 },
 "dev_eui":"647FDA0600006FC8",
 "join_eui":"647FDA3010000100"
 },
 "created_at":"2021-06-02T08:42:24.061Z",
 "updated_at":"2021-06-02T08:42:24.061Z",
 "name":"boden-07",
 "version_ids":{},
 "network_server_address":"eu1.cloud.thethings.network",
 "application_server_address":"eu1.cloud.thethings.network",
 "join_server_address":"eu1.cloud.thethings.network"
 } 

Below is the response of the second Post

{
    "ids":{
    "device_id":"boden-07",
    "application_ids":{
    "application_id":"boden-messung"
    },
    "dev_eui":"647FDA0000002FC8",
    "join_eui":"607FDA8010000100"
    },
    "created_at":"2021-06-02T08:42:24.200521997Z",
    "updated_at":"2021-06-02T08:42:24.200521997Z",
    "network_server_address":"eu1.cloud.thethings.network",
    "application_server_address":"eu1.cloud.thethings.network",
    "root_keys":{
    "app_key":{
    "key":"F2A3C66B75013D14873807577430FF73"
    }
    }
    }

Below is the response of my third Post request

{
"ids":{
"device_id":"boden-07",
"application_ids":{
"application_id":"boden-messung"
},
"dev_eui":"607FDA0000003FC8",
"join_eui":"649FDA2010000100"
},
"created_at":"2021-06-02T08:42:24.327125687Z",
"updated_at":"2021-06-02T08:42:24.327125687Z"
}

Below is the response of my fourth Post request

{
"ids":{
"device_id":"boden-07",
"application_ids":{
"application_id":"boden-messung"
},
"dev_eui":"607FDA0000003FC8",
"join_eui":"649FDA2010000100"
},
"created_at":"2021-06-02T08:42:24.464742974Z",
"updated_at":"2021-06-02T08:42:24.464742974Z"
}

Now I have another issue. In my End Devices → General Settings, there is a field “Join Server address”, which is left empty as below.

The Network layer is also left empty as below.

Application layer is unexpandable and the Join Settings are given as below.

Can we please guide me a bit, where am I making mistakes or what might be wrong? Thank you so much. My device does not join the network.

Kind regards
Imran Khan