Gateway location data for in flight geofencing

Hello,
For more than year now, I’m doing pico superpressure balloon trackers. These fly for thousands of kilometers across continents and possible even across the whole globe. I have frequency region geofencing implemented so the tracker automatically switches between the frequency bands mid flight.

The problem is, when the tracker flies across a sea or an ocean, I lose the data from that part of the flight. So I want the tracker to do some internal backlogging. Logging the data is easy, but I need to reliably know whether the tracker is in an area with good or bad coverage. One way would be to receive an ack from TTN to confirm the packet was received, but since I don’t have a receiving circuit there, I don’t want to play with that. So a better idea would be to create a geofence map which would define the perimeter of the areas with good coverage and bad or no coverage. I can do that either manualy, or generate it automatically using a script setting a perimeter of 500km (my average range) for each gateway.

The question is, is there some file/database/json with all the locations of all the TTN gateways? Like the ones you see on the TTN homepage. It could really help me.

Thanks everyone

See Coordinates of all gateways.

Just curious, what are you using as transmitter(s) and why is there no receiver circuit?

Thank you very much, it’s exactly what I needed.

Alright, until now I was using and ordinary RFM95. The old tracker looks like this:IMG_20200302_122536 InkedIMG_20200302_122525_LI

But it’s 17 grams which is huge for a picoballoon. I’ve talked to some other people who do this and discovered what a huge difference the weight makes. So I sat down and designed the new prototype for weeks. It’s not asembled yet, so here are some 3D renders:1 4

As you can see, this is much smaller. There are BGA components and 0201 passives. So I strapped it of everything that’s not absolutely required. So there is just an SX1276 with one HF bandpass filter. The whole tracker (without battery) should weight around 1.1 grams.

But since I controll the whole hardware, I can add the RF switch and filter and the tracker will also be able to receive. I’m also thinking about adding an LF output (since the SX1276 can output and input both 433 and 900MHz) for the Chinese TTN frequnecy. Since this is my first work with custom bandpass filters and these smalls compoents, I didn’t include that yet. Maybe in the futute.

1 Like

Have you considered the SAM R34 from Microchip or the STM32WLE5J8/JB/JC from ST? That could potentially reduce the component count even more.

Yes, of course I did. Neither of those are available yet, right? They’re just in production.

Main problem is that I’m years old (so elementary school) and I have little coding experience. All my trackers are still arduino compatible but for those chips I would have to use more advanced IDEs and write create my own libraries for writing the registers and so on.

I realize that I will have to do this someday, I’m just taking little steps. I don’t want a tracker with functional HW but it won’t work because I wouldn’t be able to code it.

The microchip devices have been available for over a year. Several boards with them have been released already. And there is a reference stack available, so no need to start from scratch,

Alright, I’ll look into it. Thank you for the suggestion.

Hi everyone! We’ve been using this data inside our API for quite some time and our systems kinda got dependent on it. Recently, these systems stopped working.

One of our team members who programmed the calls said that it’s likely that CORS was disabled for this data. Does anyone know why that would happen and how to fix it? Thanks!

The sx1276 and all its alternatives can already both transmit and receive.

What you need is software not hardware

a better idea would be to create a geofence map which would define the perimeter of the areas with good coverage and bad or no coverage.

That won’t work, as you don’t know if they are actually in operation at any given time

I need to reliably know whether the tracker is in an area with good or bad coverage.

Look at how ADR works; you might not be able to use it literally without modification, but the basic idea could work. Especially as you’d only want to do batch uploads of saved data when you had sufficient connectivity for a higher data rate, which is exactly what ADR is intended to determine.

Yes, but at the time there was no RF switch implemented so I couldn’t code my way trough that. The SX1276 has 5 distinct RF pins and only 1 of them was used at the time.

It’s supposed to give just a very rough estimate of this. It’s okay if it’s in covered areas and thinks it’s not. But it should not make mistakes the opposite way. The problem is that the majority of stations are in US and EU and are “covering” each others receiving areas. We now have a bitmap where the gateways are stored and each pixels contains the latitude and longitude. From this the probe can determine if it’s in a covered area (albeit only in a previous snapshot of gateways but that should be okay if we update it with each new FW revision).

Huh. From what I’ve seen the ADR process can quite some time to actually calibrate and work well. When the velocity of the transmitter can be up to 50 m/s and the angular velocity can be around 90 deg/s I don’t know when the calibration process can occur and how accurate it will be.

That’s precisely why it won’t work. Just because a gateway is on a map, or even was in operation when you launched, doesn’t mean it still is when you fly into its horizon.

From what I’ve seen the ADR process can quite some time to actually calibrate and work well. When the velocity of the transmitter can be up to 50 m/s and the angular velocity can be around 90 deg/s I don’t know when the calibration process can occur and how accurate it will be.

That’s why you might need to use the idea rather than the existing implementation.

But first you should have an antenna omnidirectional enough for rotation not to matter, especially as at the longer range spreading factors you’d see substantial rotation during a single packet.

In terms of ADR-like schemes, one thing you could do is operate in two modes, possibly even by being two virtual nodes. One would periodically transmit essential current data at the slowest spreading factor legal for the location. The other could attempt a more aggressive ADR at a faster SF. Possibly you could just periodically send out a solitary SF7 confirmed uplink - but be sure not to implement retry (which is what makes confirmed normally unwise). If you get an ack, then you blast out saved data. If you don’t, you wait ten minutes and try again.

Or something like that.

Ok, thanks for the info. We’ll try something like this.



Is there anyone who can advise on this?

What data do you refer to? It is probably obvious to you, but my crystal ball wore out long ago.

And you probably can rely on an answer appearing within hours. Given it a day or two…

Yes so first we have a gateway map where we can filter gateways trough their active dates and change thier radiuses where they can receive to better visualize the covered area. We also have our own public API which has the bitmap generator a mentioned earlier. The request documentation is not made yet though.

Yes of course I expect that, but the question got under 3 other posts so I wanted to keep it visible in the last one.

There should be no issue with CORS for a plain request from your PHP script gateway.php but it is reporting 502 bad gateway.

There are a number of questions about the architecture of the app but the main one would be regarding the gateway list - it doesn’t change fast enough to warrant requesting it for each access of your app - it may be you are being blocked due to too many requests. Perhaps look at grabbing the data every 6 or 12 hours, parsing it and turning it in to a static file.

If you have a repro we can see the code base, I’d be happy to look in to it further.

I see. Weird thing is that this worked before but just stopped now.

Okay, that makes sense. We were talking about caching this inside our API, but someone raised concerns about whether this data isn’t copyrighted. I suppose it isn’t at least for these purposes, or do you know something else?

Sure. We dont have a public repo but there are only two files for this. This HTML and then the gateways.php file:

<?php

if($_SERVER['REQUEST_METHOD'] === 'GET') {

    $ch = curl_init();

    if($ch === false) {
        throw new Exception('failed to initialize');
    }

    curl_setopt($ch, CURLOPT_URL, 'https://www.thethingsnetwork.org/gateway-data/');
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);

    $response = curl_exec($ch);

    if ($response === false) {
        throw new Exception(curl_error($ch), curl_errno($ch));
    }

    echo $response;
}
?>

I hope that this format is okay.

I’d not consider this copyright (but I’m no lawyer), apart from fair use, every data point is someone’s gateway so this is a compilation of public data and TTN is a community after all and you aren’t using it for commercial gain etc etc

The most obvious test is, can you run your script on a machine not on the same network as your proxy. I can run it on one of my web servers, so there is either an issue with yours or, as suggested and not unreasonably, TTN are rate limiting.

You can literally fix this now by replacing the gateways.php with the output from using your browser and then figure out an update scheme in the morning - cron based preferably, then there is no delay if a script sees an older file and goes to update it on the fly.

2 Likes

Thanks very much for the suggestions! I will write here about our results when we have them.