# Server getting data from TTN (lots of "connections) with same counter)

**URL:** <https://www.thethingsnetwork.org/forum/t/server-getting-data-from-ttn-lots-of-connections-with-same-counter/40719>\
**Category:** MikroTik Gateways\
**Created:** [October 20, 2020, 7:18pm UTC](https://www.thethingsnetwork.org/forum/t/server-getting-data-from-ttn-lots-of-connections-with-same-counter/40719 "2020-10-20T19:18:51Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![mvcorrea](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/mvcorrea/32/38295_2.png) [@mvcorrea](https://www.thethingsnetwork.org/forum/u/mvcorrea)\
**Post date:** [October 20, 2020, 7:18pm UTC](https://www.thethingsnetwork.org/forum/t/server-getting-data-from-ttn-lots-of-connections-with-same-counter/40719/1 "2020-10-20T19:18:51Z")

</div>

A question here,  
Gateway([mikrotik](https://mikrotik.com/product/wap_lr9_kit)) and sensor([rak7200](https://store.rakwireless.com/products/rak7200-lpwan-tracker)) side by side.  
When trying to get some data via node.js…(see code)

```
const mqtt = require("mqtt");
const Datastore = require("nedb");
const db = new Datastore({ filename: "./tracker.db", autoload: true });
const opts = { 
  auth: { 
          username: "AAAAAA", 
          password: "ttn-account-v2.BBBBBB",
          clean: true,
          port: 1883,
  },
  topic: "CCCCCC/devices/DDDDDD/up",
  host: "mqtt://eu.thethings.network",
};

const client = mqtt.connect(opts.host, opts.auth);
client.on("connect", () => {
    client.subscribe(opts.topic);
});

client.on("message", (topic, msg) => {
    const stringBuf = msg && msg.toString("utf-8");
    const jsonMsg = JSON.parse(stringBuf);
    db.insert(jsonMsg, (err, doc) => {
	    if (err) console.log("error: " + err);
	    else console.log(`inserted [${doc.dev_id}] with counter [${doc.counter}.${doc.confirmed?"confirmed":"unconfirmed"}] `)
    });
    //console.log(jsonMsg);
});

```

everything is working properly, however I got plenty of entrys with same counter number!  
The application was created asking for confirmation!

```
inserted [tracker01] with counter [1662.confirmed] 
inserted [tracker01] with counter [1662.confirmed] 
inserted [tracker01] with counter [1662.confirmed] 
inserted [tracker01] with counter [1662.confirmed] 
inserted [tracker01] with counter [1662.confirmed] 
inserted [tracker01] with counter [1662.confirmed] 
inserted [tracker01] with counter [1662.confirmed] 
inserted [tracker01] with counter [1662.confirmed] 
inserted [tracker01] with counter [1663.confirmed] 
inserted [tracker01] with counter [1663.confirmed] 
inserted [tracker01] with counter [1663.confirmed] 
inserted [tracker01] with counter [1663.confirmed] 
inserted [tracker01] with counter [1663.confirmed] 
inserted [tracker01] with counter [1664.confirmed] 
inserted [tracker01] with counter [1664.confirmed] 
inserted [tracker01] with counter [1664.confirmed] 
inserted [tracker01] with counter [1664.confirmed] 
inserted [tracker01] with counter [1664.confirmed] 
inserted [tracker01] with counter [1664.confirmed] 
inserted [tracker01] with counter [1664.confirmed] 
inserted [tracker01] with counter [1664.confirmed] 
inserted [tracker01] with counter [1665.confirmed] 
inserted [tracker01] with counter [1665.confirmed] 
inserted [tracker01] with counter [1665.confirmed] 
inserted [tracker01] with counter [1665.confirmed] 
inserted [tracker01] with counter [1665.confirmed] 
inserted [tracker01] with counter [1666.confirmed] 
inserted [tracker01] with counter [1666.confirmed] 
inserted [tracker01] with counter [1666.confirmed] 
inserted [tracker01] with counter [1666.confirmed] 
inserted [tracker01] with counter [1666.confirmed] 
inserted [tracker01] with counter [1666.confirmed] 
inserted [tracker01] with counter [1666.confirmed] 
inserted [tracker01] with counter [1666.confirmed] 

```

its supposed to be like this?  
I need to keep (db write) only one of those (with the same counter)?  
Anyway to minimize those connections?  
regards,

full package example (using default RAK7200 decoder)

```
{
  app_id: 'test_gps_tracXXXX',
  dev_id: 'trackXXX',
  hardware_serial: 'AC1F09FFFXXXXXX',
  port: 8,
  counter: 1650,
  confirmed: true,
  payload_raw: 'CAIBeANxAAgAGAO2BYYAPABl/7CgXXXXXXXXXXX=',
  payload_fields: {
    acceleration_x: '0.008g',
    acceleration_y: '0.024g',
    acceleration_z: '0.95g',
    battery: '3.76V',
    gyroscope_x: '0.6°/s',
    gyroscope_y: '1.01°/s',
    gyroscope_z: '-0.67°/s',
    magnetometer_x: '43.5μT',
    magnetometer_y: '66.75μT',
    magnetometer_z: '-9.3μT'
  },
  metadata: {
    time: '2020-10-20T17:32:29.276946973Z',
    frequency: 918.2,
    modulation: 'LORA',
    data_rate: 'SF7BW125',
    airtime: 97536000,
    coding_rate: '4/5',
    gateways: [[Object] ]
  }
}
```

---

<div class="post-metadata">

**Author:** ![cslorabox](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/cslorabox/32/35697_2.png) [@cslorabox](https://www.thethingsnetwork.org/forum/u/cslorabox)\
**Post date:** [October 20, 2020, 7:36pm UTC](https://www.thethingsnetwork.org/forum/t/server-getting-data-from-ttn-lots-of-connections-with-same-counter/40719/2 "2020-10-20T19:36:11Z")

</div>

> [@mvcorrea](#):
>
> Gateway([mikrotik](https://mikrotik.com/product/wap_lr9_kit)) and sensor([rak7200](https://store.rakwireless.com/products/rak7200-lpwan-tracker)) side by side.

That’s known to be a bad idea, it will at minimum cause false duplicate uplinks on multiple frequencies due to front end overload, and may also cause corruption of even the packet on the correct frequency

> inserted [tracker01] with counter [1662.confirmed]

None of this data is coming from your actual JSON message received, but only from some mysterious document variables. You should log out something based on the actual received message instead, or as well

> The application was created asking for confirmation!

You should not do that.

First, if confirmed traffic worked, it would burn through your device’s downlink allowance in no time.

But the protocol of confirmed uplink doesn’t really work very well anyway, due to design defects - if it misses the ack, it will never get another until it gives up on retrying to send that data item and sends a new one instead.

---

<div class="post-metadata">

**Author:** ![mvcorrea](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/mvcorrea/32/38295_2.png) [@mvcorrea](https://www.thethingsnetwork.org/forum/u/mvcorrea)\
**Post date:** [October 20, 2020, 8:29pm UTC](https://www.thethingsnetwork.org/forum/t/server-getting-data-from-ttn-lots-of-connections-with-same-counter/40719/3 "2020-10-20T20:29:33Z")

</div>

@cslorabox thanks for your promptly response!

I forgot to say I am in testing here discovering the possibilities!!  
learning things here…

- keep devices far from each other (done)
- create applications without confirmation (anyway to disable confirmation within the app?)

but on the second topic you say “None of this data is coming from your actual JSON message received”, that one i didint got it! its everything I got from the ttn!! Whats/Where is the “actual received message”?

---

<div class="post-metadata">

**Author:** ![cslorabox](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/cslorabox/32/35697_2.png) [@cslorabox](https://www.thethingsnetwork.org/forum/u/cslorabox)\
**Post date:** [October 20, 2020, 8:33pm UTC](https://www.thethingsnetwork.org/forum/t/server-getting-data-from-ttn-lots-of-connections-with-same-counter/40719/4 "2020-10-20T20:33:26Z")

</div>

> [@mvcorrea](#):
>
> create applications without confirmation (anyway to disable confirmation within the app?)

Uplink confirmation is a behavior of your node’s firmware. Don’t set that.

Downlink confirmation is a behavior of your downlink request, don’t set that either.

> “None of this data is coming from your actual JSON message received”, that one i didint got it! its everything I got from the ttn!!

No, your log messages only logs document variables, it doesn’t log anything it actually got from the method call.

> console.log(`inserted [${doc.dev_id}] with counter [${doc.counter}.${doc.confirmed?"confirmed":"unconfirmed"}] `)

None of this comes from the

> client.on(“message”, (topic, msg)

So as a result you just see the same thing printed over an over each time you _get_ a message, not but reflecting any details _of_ that message.

---

<div class="post-metadata">

**Author:** ![mvcorrea](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/mvcorrea/32/38295_2.png) [@mvcorrea](https://www.thethingsnetwork.org/forum/u/mvcorrea)\
**Post date:** [October 20, 2020, 9:26pm UTC](https://www.thethingsnetwork.org/forum/t/server-getting-data-from-ttn-lots-of-connections-with-same-counter/40719/5 "2020-10-20T21:26:48Z")

</div>

@cslorabox, I dont know if I got you!  
changing the script a little!!

```
const moment = require("moment-timezone");

const gps = (json) => {
  return json.payload_fields.latitude ? "with GPS" : "without GPS";
};

const time = (json) => {
  let time = json.metadata.gateways[0].time
    ? json.metadata.gateways[0].time	// gateway time
    : json.metadata.time; // server time
  return moment	
    .utc(time)
    .tz("America/Fortaleza")
    .format("YYYY-MM-DD HH:mm:ss.SSS");
};

// and changing the mentioned line

 console.log(
    `inserted [${doc.dev_id}] with counter [${doc.counter}.${
      doc.confirmed ? "confirmed" : "unconfirmed"
    }] ${gps(jsonMsg)}, with time: ${time(jsonMsg)}`
 );

```

I got different timestamps on each message! I think They are not the same!  
those messages are the output of the method call! isn’t it? If not, how to log the right messages?  
You are right confirmation is device specific! I am gonna change it right now!  
I forgot to mention I setup the device to send messages each 180 secs (20 samples/hour)  
the jsonMsg came from the client.on(“message”…) event! and is the same as msg (after I parse it as json).  
I really dont know What I am missing here!  
output:

```
inserted [tracker01] with counter [1693.confirmed] with GPS, with time: 2020-10-20 18:06:24.554
inserted [tracker01] with counter [1693.confirmed] with GPS, with time: 2020-10-20 18:06:29.565
inserted [tracker01] with counter [1693.confirmed] with GPS, with time: 2020-10-20 18:06:34.739
inserted [tracker01] with counter [1693.confirmed] with GPS, with time: 2020-10-20 18:06:40.751
inserted [tracker01] with counter [1693.confirmed] with GPS, with time: 2020-10-20 18:06:45.090
inserted [tracker01] with counter [1693.confirmed] with GPS, with time: 2020-10-20 18:06:51.103
inserted [tracker01] with counter [1694.confirmed] with GPS, with time: 2020-10-20 18:10:44.433
inserted [tracker01] with counter [1694.confirmed] with GPS, with time: 2020-10-20 18:10:48.440
inserted [tracker01] with counter [1694.confirmed] with GPS, with time: 2020-10-20 18:10:54.190
inserted [tracker01] with counter [1694.confirmed] with GPS, with time: 2020-10-20 18:11:01.203
inserted [tracker01] with counter [1694.confirmed] with GPS, with time: 2020-10-20 18:11:08.448
```

---

<div class="post-metadata">

**Author:** ![arjanvanb](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/arjanvanb/32/46626_2.png) [@arjanvanb](https://www.thethingsnetwork.org/forum/u/arjanvanb)\
**Post date:** [October 20, 2020, 10:58pm UTC](https://www.thethingsnetwork.org/forum/t/server-getting-data-from-ttn-lots-of-connections-with-same-counter/40719/6 "2020-10-20T22:58:45Z")

</div>

> [@mvcorrea](#):
>
> I got different timestamps on each message! I think They are not the same!

That’s because, for _confirmed_ uplinks, your device is repeating the very same uplink (with the very same counter value) until it receives a downlink with an acknowledgement, or until it gives up. And like already noted, apparently missing the first acknowledgement will never make it receive one for subsequent retries:

> [@cslorabox](#):
>
> due to design defects - if it misses the ack, it will never get another until it gives up on retrying to send that data item and sends a new one instead.

If you would print the `is_retry` attribute you would see that TTN has recognized it was a retry of an earlier message that was _exactly_ the same. And along with that very same message you’re getting _different_ details for things like the timestamp, the list of gateway(s) that received the retry, and the signal quality for each gateway.

Apparently your device has problems receiving downlinks, hence does not get the acknowledgement for confirmed uplinks, and _assumes_ its uplink was not even received by any gateway.

---

<div class="post-metadata">

**Author:** ![mvcorrea](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/mvcorrea/32/38295_2.png) [@mvcorrea](https://www.thethingsnetwork.org/forum/u/mvcorrea)\
**Post date:** [October 21, 2020, 1:23am UTC](https://www.thethingsnetwork.org/forum/t/server-getting-data-from-ttn-lots-of-connections-with-same-counter/40719/8 "2020-10-21T01:23:24Z")

</div>

I dont have access to the is\_retry attribute (is it device specific?), but makes sense!

looking in the ttn console\>data I noticed this!

![image](https://www.thethingsnetwork.org/forum/uploads/default/original/3X/e/1/e18b5987acbafc7d8074c38b11e9fe07180817f3.png)

its possible to verify that the first msg is confirmed followed by retry confirmed msgs!  
thats the default behavior? Could you please point me in the direction to understand those type of messages?

---

<div class="post-metadata">

**Author:** ![mvcorrea](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/mvcorrea/32/38295_2.png) [@mvcorrea](https://www.thethingsnetwork.org/forum/u/mvcorrea)\
**Post date:** [October 21, 2020, 1:45am UTC](https://www.thethingsnetwork.org/forum/t/server-getting-data-from-ttn-lots-of-connections-with-same-counter/40719/9 "2020-10-21T01:45:21Z")

</div>

Confirmed here, on the first message I dont have the is\_retry attribute! followed by many of those! If the first message was confirmed why to retry confirmation on the followed ones?  
As requested the printed json!!

```
{
  app_id: 'test_gps_tracker_01',
  dev_id: 'tracker01',
  hardware_serial: 'AC1F09FFFE00375A',
  port: 8,
  counter: 1747,
  confirmed: true, <--- no is_rety attribute
  payload_raw: 'AYj/bqL6H3kAAtoIAgF1A3EAGAAqA7wFhgBOAGT/wAkCGdcKAhHBCwL9bA==',
  payload_fields: {
    acceleration_x: '0.024g',    
    acceleration_y: '0.042g',
    acceleration_z: '0.956g',
    altitude: '7.3m',
    battery: '3.73V',
    gyroscope_x: '0.78°/s',
    gyroscope_y: '1°/s',
    gyroscope_z: '-0.64°/s',
    latitude: '-3.7214°',
    longitude: '-38.5159°',
    magnetometer_x: '66.15μT',
    magnetometer_y: '45.45μT',
    magnetometer_z: '-6.6μT'
  },
  metadata: {
    time: '2020-10-21T01:26:14.33970477Z',
    frequency: 917.8,
    modulation: 'LORA',
    data_rate: 'SF7BW125',
    airtime: 112896000,
    coding_rate: '4/5',
    gateways: [[Object] ]
  }
}
{
  app_id: 'test_gps_tracker_01',
  dev_id: 'tracker01',
  hardware_serial: 'AC1F09FFFE00375A',
  port: 8,
  counter: 1747,
  confirmed: true,  
  is_retry: true,
  payload_raw: 'AYj/bqL6H3kAAtoIAgF1A3EAGAAqA7wFhgBOAGT/wAkCGdcKAhHBCwL9bA==',
  payload_fields: {
    acceleration_x: '0.024g',
    acceleration_y: '0.042g',
    acceleration_z: '0.956g',
    altitude: '7.3m',
    battery: '3.73V',
    gyroscope_x: '0.78°/s',
    gyroscope_y: '1°/s',
    gyroscope_z: '-0.64°/s',
    latitude: '-3.7214°',
    longitude: '-38.5159°',
    magnetometer_x: '66.15μT',
    magnetometer_y: '45.45μT',
    magnetometer_z: '-6.6μT'
  },
  metadata: {
    time: '2020-10-21T01:26:18.350087725Z',
    frequency: 917.4,
    modulation: 'LORA',
    data_rate: 'SF7BW125',
    airtime: 112896000,
    coding_rate: '4/5',
    gateways: [[Object] ]
  }
}
{
  app_id: 'test_gps_tracker_01',
  dev_id: 'tracker01',
  hardware_serial: 'AC1F09FFFE00375A',
  port: 8,
  counter: 1747,
  confirmed: true,
  is_retry: true,
  payload_raw: 'AYj/bqL6H3kAAtoIAgF1A3EAGAAqA7wFhgBOAGT/wAkCGdcKAhHBCwL9bA==',
  payload_fields: {
    acceleration_x: '0.024g',
    acceleration_y: '0.042g',
    acceleration_z: '0.956g',
    altitude: '7.3m',
    battery: '3.73V',
    gyroscope_x: '0.78°/s',
    gyroscope_y: '1°/s',
    gyroscope_z: '-0.64°/s',
    latitude: '-3.7214°',
    longitude: '-38.5159°',
    magnetometer_x: '66.15μT',
    magnetometer_y: '45.45μT',
    magnetometer_z: '-6.6μT'
  },
  metadata: {
    time: '2020-10-21T01:26:24.459928828Z',
    frequency: 917,
    modulation: 'LORA',
    data_rate: 'SF8BW125',
    airtime: 205312000,
    coding_rate: '4/5',
    gateways: [[Object] ]
  }
}
```

---

<div class="post-metadata">

**Author:** ![cslorabox](https://www.thethingsnetwork.org/forum/user_avatar/www.thethingsnetwork.org/cslorabox/32/35697_2.png) [@cslorabox](https://www.thethingsnetwork.org/forum/u/cslorabox)\
**Post date:** [October 21, 2020, 1:32pm UTC](https://www.thethingsnetwork.org/forum/t/server-getting-data-from-ttn-lots-of-connections-with-same-counter/40719/10 "2020-10-21T13:32:46Z")

</div>

> [@mvcorrea](#):
>
> its possible to verify that the first msg is confirmed followed by retry confirmed msgs!  
> thats the default behavior? Could you please point me in the direction to understand those type of messages?

This is exactly what happens when the ack of the first attempt is missed. No more ack will ever be generated for that message number, so transmission retries will continue until the node gives up and moves on to the next message number.

This colossal waste of airtime and battery resulting from the bad design of confirmed uplink in LoRaWAN is **why you should not use confirmed uplink**
