Hi all, trying to register a lot of devices via csv import function.
I explicitly specify the id of the devices in my csv file. But it will always take the device_eui as the id.
If the devices are registered with the JSON import function then it will allow the device_id as such…
Is there a way to also properly do this via CSV files?
Hello,
Same issue for me and one year later. doing it in json it is working but not in csv which is more convenient…
Thanks for reporting! We’re looking into this.
Hi @GFO_Tests92,
Like @kschiffer said we are indeed looking into this. The main trouble for me regarding this is that I cannot seem to replicate the behavior that you are describing.
I’m creating devices via csv while specifying both their device ID and devEUI, the later does not replace the deviceID on my tests.
Hope you don’t mind but there are a few things that would help me out finding and fixing this.
My Excel doc for preparation of creds for deployment works with CSV.
The field headers have to be all lower case and exactly as per the template.
And someone decided that commas have a dot above them - it may be some sort of accent for a regional dialect, but as long as you use:
and not
all should be good.
Recently it was added the parser which accepts commas as separators.
There might still be problems if you for example use a spreadsheet in the Portuguese language, don’t know if its exclusive to the Brazilian version but decimal cases are separated by commas instead of dots. Instead of being 4000.00 it is 4000,00, this might cause parsing problems on the TTN with a field like hardware_version. So the usage of semicolons is still recommended.
You can in Excel specify the decimal separator - File > Options > Advanced
I tried again and same issue, you can find my file example in attachment
Please let me know if I am doing something wrong.
Posting text as an image - I can’t try that out - this is a cardinal sin on this forum.
And not posting the error message!
@GFO_Tests92 I don’t see an issue with your screenshot, but as @descartes says, we can’t test this way.
Please file an issue here: Sign in to GitHub · GitHub. We made this issue template precisely to get enough information; in this case, to reproduce. Without an issue we can’t plan any fixes and people need to search both the forums and GitHub for help, so we only work with issues. Thanks.
Test is passing when I take the relevant bits from your screenshot:
diff --git a/pkg/devicetemplates/testdata/input_device_id.csv b/pkg/devicetemplates/testdata/input_device_id.csv
new file mode 100644
index 000000000..59d0d9a50
--- /dev/null
+++ b/pkg/devicetemplates/testdata/input_device_id.csv
@@ -0,0 +1,2 @@
+id;dev_eui
+5on102a;353131397837660A
diff --git a/pkg/devicetemplates/ttscsv_test.go b/pkg/devicetemplates/ttscsv_test.go
index f5785ae19..3716ad706 100644
--- a/pkg/devicetemplates/ttscsv_test.go
+++ b/pkg/devicetemplates/ttscsv_test.go
@@ -43,6 +43,8 @@ var (
csvInvalidDevEUI string
//go:embed testdata/generate_device_id.csv
csvGenerateDeviceID string
+ //go:embed testdata/input_device_id.csv
+ csvInputDeviceID string
//go:embed testdata/appeui.csv
csvAppEUI string
//go:embed testdata/comma_separated.csv
@@ -160,6 +162,26 @@ func TestTTSCSVConverter(t *testing.T) {
},
nExpect: 1,
},
+ {
+ name: "https://www.thethingsnetwork.org/forum/t/csv-file-import-device-id-issue/57654",
+ reader: bytes.NewBufferString(csvInputDeviceID),
+ converter: GetConverter("the-things-stack-csv"),
+ validateError: func(a *assertions.Assertion, err error) {
+ a.So(err, should.BeNil)
+ },
+ validateResult: func(a *assertions.Assertion, templates []*ttnpb.EndDeviceTemplate, count int) {
+ if !a.So(len(templates), should.Equal, count) {
+ t.FailNow()
+ }
+ dev := templates[0]
+ a.So(dev.EndDevice.Ids.DeviceId, should.Equal, "5on102a")
+ a.So(ttnpb.RequireFields(dev.FieldMask.Paths,
+ "ids.device_id",
+ "ids.dev_eui",
+ ), should.BeNil)
+ },
+ nExpect: 1,
+ },
{
name: "AppEUI",
reader: bytes.NewBufferString(csvAppEUI),
I’d note that the brand_id column is probably not what the OP thinks it’s for - given that its a replica of the id. Is that likely to cause an issue with lookups in the device repository.
But getting text and the error message posted here may just allow the problem to be solved and not end up with a GitHub issue that’s a non-issue.
Hello,
Sorry for my last communication, in fact not smart from my side to give you the details.
To be able to reproduce the issue, I did :
Converting end device templates…
{
“end_device”: {
“ids”: {
“device_id”: “eui-353131397037820a”,
“dev_eui”: “353131397037820A”,
“join_eui”: “70B3D5FFFE0B5505”
},
“name”: “5on102b”,
“version_ids”: {
“brand_id”: “5on102b”
},
“lorawan_version”: “MAC_V1_0_3”,
“lorawan_phy_version”: “PHY_V1_0_3_REV_A”,
“frequency_plan_id”: “EU_863_870_TTN”,
“supports_join”: true,
“root_keys”: {
“app_key”: {
“key”: “F832899D7B0209E1E493901BFB20AE7D”
}
}
},
“field_mask”: “ids.dev_eui,ids.join_eui,name,frequency_plan_id,lorawan_version,lorawan_phy_version,version_ids.brand_id,root_keys.app_key.key,supports_join,ids.device_id”
}
Creating end devices…
{
“ids”: {
“device_id”: “eui-353131397037820a”,
“application_ids”: {
“application_id”: “desks”
},
“dev_eui”: “353131397037820A”,
“join_eui”: “70B3D5FFFE0B5505”
},
“created_at”: “2022-11-22T06:33:54.113521Z”,
“updated_at”: “2022-11-22T06:33:54.113521Z”,
“version_ids”: {
“brand_id”: “5on102b”
},
“lorawan_version”: “MAC_V1_0_3”,
“lorawan_phy_version”: “PHY_V1_0_3_REV_A”,
“frequency_plan_id”: “EU_863_870_TTN”,
“supports_join”: true,
“root_keys”: {
“app_key”: {
“key”: “F832899”
}
},
“application_server_address”: “eu1.cloud.thethings.network”,
“network_server_address”: “eu1.cloud.thethings.network”,
“name”: “5on102b”,
“claim_authentication_code”: {
“value”: “3857A3CF”
},
“join_server_address”: “eu1.cloud.thethings.network”
}
Import operation complete
Please let me know if you need more details or other tests
Thanks for your support
@GFO_Tests92 just like Johan said, I think its best to create an issue on TTS create Issue
I tried to replicate what you did on these steps and the ID was generated properly, there is a series of steps on the issue creation that will help with the debugging on our end.