No Fields?

I had some really useful help on this forum recently when I was building a LoRaWAN tracker based on an Arduino Pro Mini. I’ve since built a few which worked well but today I hit a new problem. I have used a new type of GPS module and for some reason the packets of data do not include any fields for lat, long or attitude. The design of the tracker has not changed except for the GPS. The previous GPS modules were based on the NEO 6M but this new one is based on an equivalent of the NEO 8M. This may or may not be important.

The tracker is successfully getting a fix. By adding some lines to the code I can get it to print out lat, long, altitude and number of satellites and all the values are correct.

The problem is why aren’t these being decoded?

Here is the relevant (I think) code which handles the coords. The print statements near the beginning are for de-bugging only.

// For one second we parse GPS data and report some key values

  for (unsigned long start = millis(); millis() - start < 1000;) {
    while (ss.available()) {
      char c = ss.read();
      Serial.write(c); // uncomment this line if you want to see the GPS data flowing
      if (gps.encode(c)) { // Did a new valid sentence come in?
        newData = true;
      }
    }
  }

 if ( newData ) {
    flat=gps.location.lat();
    flon=gps.location.lng();
    Serial.println("latitude");
    Serial.println(gps.location.lat());
    Serial.println("longitude");
    Serial.println(gps.location.lng());
    Serial.println("altitude");
    Serial.println(gps.altitude.meters());
    if (gps.altitude.isValid())
         faltitudeGPS = gps.altitude.meters();
              else
        faltitudeGPS=0;
    if (gps.satellites.isValid())
        fsatellitesGPS = gps.satellites.value();
      else
        fsatellitesGPS=0;  
    Serial.println("satellites");
    Serial.println(gps.satellites.value()); //Shows number of satellites - working!
            
}
 
  int32_t lat = flat * 10000;
  int32_t lon = flon * 10000;
  int32_t altitudeGPS = faltitudeGPS * 100;
  channel = 0x01;
  coords[0] = channel; 
  coords[1] = LPP_GPS; 
  coords[2] = lat >> 16;
  coords[3] = lat >> 8;
  coords[4] = lat;
  coords[5] = lon >> 16; 
  coords[6] = lon >> 8;
  coords[7] = lon;
  coords[8] = altitudeGPS >> 16;
  coords[9] = altitudeGPS >> 8;
  coords[10] = altitudeGPS;
  }

void do_send(osjob_t* j) {

  // Check if there is not a current TX/RX job running
  if (LMIC.opmode & OP_TXRXPEND) {
    Serial.println(F("OP_TXRXPEND, not sending"));
  } else {
    // Prepare upstream data transmission at the next possible time.
    get_coords();
    LMIC_setTxData2(1, (uint8_t*) coords, sizeof(coords), 0);
    Serial.println(F("Packet queued"));
  } 
}
1 Like

When you say decoded, do you mean by the Arduino library handling the GPS for you or when it gets to TTN application?

The code snippet just shows the gps object passing in lat & long in to two variables called flat & flon. We need to determine what is in the gps object and how the two variables are used in the uplink payload. There is no penalty for posting all the code, it won’t fill the whole screen as long as you format it as a block quote like you already have.

Can we see the detail of the output, both from the GPS, the uplink data and what you see from the decoder output, if that’s the bit you mean.

I’m not sure if this screenshot will appear or be readable, but this is what I see on the console for a device which is working. With the one which is not working there is nothing after the intitial list of bytes 0188 07 etc and instead of a list of fields it just says “no fields”.

image

But the data is there in the device because I can print them in the serial console and the values are correct. The curious thing is this software works but not with this new tracker with a different GPS.

The full code is:

 #include <lmic.h>
#include <hal/hal.h>
#include <SPI.h>
#include <TinyGPS++.h>
#include <AltSoftSerial.h>
#define LPP_GPS 136     // 3 byte lon/lat 0.0001 °, 3 bytes alt 0.01 meter

static const int RXPin = 8, TXPin = 9;
static const uint32_t GPSBaud = 9600;

TinyGPSPlus gps;
AltSoftSerial ss(RXPin, TXPin); // RX, TX

static const PROGMEM u1_t NWKSKEY[16] = { 0xA5, 0x91, 0xE2, 0x6E, 0x63, 0x8F, 0xDA, 0xEC, 0x7E, 0xAD, 0xC4, 0x03, 0x1E, 0x4D, 0xE6, 0x43 };
static const u1_t PROGMEM APPSKEY[16] = { 0x3B, 0x82, 0xA9, 0xAD, 0x67, 0xF0, 0x9B, 0xF7, 0x11, 0x44, 0x44, 0xC8, 0x8C, 0x62, 0xEC, 0xFA };
static const u4_t DEVADDR =  0x(removed); // <-- Change this address for every node!
const unsigned TX_INTERVAL = 30;  

void os_getArtEui (u1_t* buf) { }
void os_getDevEui (u1_t* buf) { }
void os_getDevKey (u1_t* buf) { }

  uint8_t coords[11];
  static osjob_t sendjob;
  static osjob_t initjob;
  uint8_t cursor = 0;
  uint8_t channel;
  
// Pin mapping

  const lmic_pinmap lmic_pins = {
  .nss = 10,
  .rxtx = LMIC_UNUSED_PIN,
  .rst = LMIC_UNUSED_PIN,
  .dio = {4, 5, 7},
};

  //TinyGPSInteger satellites;  //not sure if this necessary
  void get_coords () {
  bool newData = false;
  unsigned long chars;
  unsigned short sentences, failed;
  //float flat,flon,faltitudeGPS,fhdopGPS; //add satellites
  float flat,flon,faltitudeGPS;
  //satellites is an integer
  unsigned short int fsatellitesGPS;
  unsigned long age;

  // For one second we parse GPS data and report some key values

  for (unsigned long start = millis(); millis() - start < 1000;) {
    while (ss.available()) {
      char c = ss.read();
      Serial.write(c); // uncomment this line if you want to see the GPS data flowing
      if (gps.encode(c)) { // Did a new valid sentence come in?
        newData = true;
      }
    }
  }

 if ( newData ) {
    flat=gps.location.lat();
    flon=gps.location.lng();
    Serial.println("latitude");
    Serial.println(gps.location.lat());
    Serial.println("longitude");
    Serial.println(gps.location.lng());
    Serial.println("altitude");
    Serial.println(gps.altitude.meters());
    if (gps.altitude.isValid())
         faltitudeGPS = gps.altitude.meters();
              else
        faltitudeGPS=0;
    if (gps.satellites.isValid())
        fsatellitesGPS = gps.satellites.value();
      else
        fsatellitesGPS=0;  
    Serial.println("satellites");
    Serial.println(gps.satellites.value()); //Shows number of satellites - working!
            
}
 
  int32_t lat = flat * 10000;
  int32_t lon = flon * 10000;
  int32_t altitudeGPS = faltitudeGPS * 100;
  channel = 0x01;
  coords[0] = channel; 
  coords[1] = LPP_GPS; 
  coords[2] = lat >> 16;
  coords[3] = lat >> 8;
  coords[4] = lat;
  coords[5] = lon >> 16; 
  coords[6] = lon >> 8;
  coords[7] = lon;
  coords[8] = altitudeGPS >> 16;
  coords[9] = altitudeGPS >> 8;
  coords[10] = altitudeGPS;
  }

void do_send(osjob_t* j) {

  // Check if there is not a current TX/RX job running
  if (LMIC.opmode & OP_TXRXPEND) {
    Serial.println(F("OP_TXRXPEND, not sending"));
  } else {
    // Prepare upstream data transmission at the next possible time.
    get_coords();
    LMIC_setTxData2(1, (uint8_t*) coords, sizeof(coords), 0);
    Serial.println(F("Packet queued"));
  } 
}

  // Next TX is scheduled after TX_COMPLETE event.

void onEvent (ev_t ev) {
  Serial.print(os_getTime());
  Serial.print(": ");
  switch(ev) {
    case EV_SCAN_TIMEOUT:
      Serial.println(F("EV_SCAN_TIMEOUT"));
      break;
    case EV_BEACON_FOUND:
      Serial.println(F("EV_BEACON_FOUND"));
      break;
    case EV_BEACON_MISSED:
      Serial.println(F("EV_BEACON_MISSED"));
      break;
    case EV_BEACON_TRACKED:
      Serial.println(F("EV_BEACON_TRACKED"));
      break;
    case EV_JOINING:
      Serial.println(F("EV_JOINING"));
      break;
    case EV_JOINED:
      Serial.println(F("EV_JOINED"));

      // Disable link check validation (automatically enabled
      // during join, but not supported by TTN at this time).
      LMIC_setLinkCheckMode(0);
      break;
    case EV_RFU1:
      Serial.println(F("EV_RFU1"));
      break;
    case EV_JOIN_FAILED:
      Serial.println(F("EV_JOIN_FAILED"));
      break;
    case EV_REJOIN_FAILED:
      Serial.println(F("EV_REJOIN_FAILED"));
      break;
      break;
    case EV_TXCOMPLETE:
      Serial.println(F("EV_TXCOMPLETE (includes waiting for RX windows)"));
      if (LMIC.txrxFlags & TXRX_ACK)
        Serial.println(F("Received ack"));
      if (LMIC.dataLen) {
        Serial.println(F("Received "));
        Serial.println(LMIC.dataLen);
        Serial.println(F(" bytes of payload"));
      }
      // Schedule next transmission
      os_setTimedCallback(&sendjob, os_getTime()+sec2osticks(TX_INTERVAL), do_send);
      break;
    case EV_LOST_TSYNC:
      Serial.println(F("EV_LOST_TSYNC"));
      break;
    case EV_RESET:
      Serial.println(F("EV_RESET"));
      break;
    case EV_RXCOMPLETE:
      // data received in ping slot
      Serial.println(F("EV_RXCOMPLETE"));
      break;
    case EV_LINK_DEAD:
      Serial.println(F("EV_LINK_DEAD"));
      break;
    case EV_LINK_ALIVE:
      Serial.println(F("EV_LINK_ALIVE"));
      break;
    default:
      Serial.println(F("Unknown event"));
      break;
  }
}

void setup()
{
  Serial.begin(115200);
  Serial.println(F("Starting"));
  ss.begin(GPSBaud);
  // LMIC init
  os_init();
  // Reset the MAC state. Session and pending data transfers will be discarded.
    LMIC_reset();
    // Set static session parameters. Instead of dynamically establishing a session
    // by joining the network, precomputed session parameters are be provided.
    #ifdef PROGMEM
    // On AVR, these values are stored in flash and only copied to RAM
    // once. Copy them to a temporary buffer here, LMIC_setSession will
    // copy them into a buffer of its own again.
    uint8_t appskey[sizeof(APPSKEY)];
    uint8_t nwkskey[sizeof(NWKSKEY)];
    memcpy_P(appskey, APPSKEY, sizeof(APPSKEY));
    memcpy_P(nwkskey, NWKSKEY, sizeof(NWKSKEY));
    LMIC_setSession (0x1, DEVADDR, nwkskey, appskey);
    #else
    // If not running an AVR with PROGMEM, just use the arrays directly
    LMIC_setSession (0x1, DEVADDR, NWKSKEY, APPSKEY);
    #endif

    #if defined(CFG_eu868)
    // Set up the channels used by the Things Network, which corresponds
    // to the defaults of most gateways. Without this, only three base
    // channels from the LoRaWAN specification are used, which certainly
    // works, so it is good for debugging, but can overload those
    // frequencies, so be sure to configure the full frequency range of
    // your network here (unless your network autoconfigures them).
    // Setting up channels should happen after LMIC_setSession, as that
    // configures the minimal channel set.
    // NA-US channels 0-71 are configured automatically
    LMIC_setupChannel(0, 868100000, DR_RANGE_MAP(DR_SF12, DR_SF7),  BAND_CENTI);      // g-band
    LMIC_setupChannel(1, 868300000, DR_RANGE_MAP(DR_SF12, DR_SF7B), BAND_CENTI);      // g-band
    LMIC_setupChannel(2, 868500000, DR_RANGE_MAP(DR_SF12, DR_SF7),  BAND_CENTI);      // g-band
    LMIC_setupChannel(3, 867100000, DR_RANGE_MAP(DR_SF12, DR_SF7),  BAND_CENTI);      // g-band
    LMIC_setupChannel(4, 867300000, DR_RANGE_MAP(DR_SF12, DR_SF7),  BAND_CENTI);      // g-band
    LMIC_setupChannel(5, 867500000, DR_RANGE_MAP(DR_SF12, DR_SF7),  BAND_CENTI);      // g-band
    LMIC_setupChannel(6, 867700000, DR_RANGE_MAP(DR_SF12, DR_SF7),  BAND_CENTI);      // g-band
    LMIC_setupChannel(7, 867900000, DR_RANGE_MAP(DR_SF12, DR_SF7),  BAND_CENTI);      // g-band
    LMIC_setupChannel(8, 868800000, DR_RANGE_MAP(DR_FSK,  DR_FSK),  BAND_MILLI);      // g2-band
    // TTN defines an additional channel at 869.525Mhz using SF9 for class B
    // devices' ping slots. LMIC does not have an easy way to define set this
    // frequency and support for class B is spotty and untested, so this
    // frequency is not configured here.
    #elif defined(CFG_us915)

    // NA-US channels 0-71 are configured automatically
    // but only one group of 8 should (a subband) should be active
    // TTN recommends the second sub band, 1 in a zero based count.
    // https://github.com/TheThingsNetwork/gateway-conf/blob/master/US-global_conf.json
    LMIC_selectSubBand(1);
    #endif

    // Disable link check validation
    LMIC_setLinkCheckMode(0);
    
    // TTN uses SF9 for its RX2 window.
    LMIC.dn2Dr = DR_SF9;

    // Set data rate and transmit power for uplink (note: txpow seems to be ignored by the library)
    LMIC_setDrTxpow(DR_SF7,14);

    // Start job
    do_send(&sendjob);
}

void loop() {
  os_runloop_once();
}

So, we have some of the pieces of the puzzle. Having the bytes that arrive for the not-working one would be exceptionally useful, as would the serial debug output for a couple of transmit cycles.

Then we can see from the code what’s being stuffed in to the payload from the GPS and figure out why it’s not decodable.

This is a screen print from the device which isn’t working.

image

I have some shots from the serial port taken earlier but in both cases the device did not have a fix and the de-bugging print outs were not added. I added these later but in both cases the lat, long etc were of simliar values so both devices were getting a fix and the software could extract the values.

I will repeat this tomorrow and post new screenshots with the debug print outs and with both having a fix. This is a bit difficult to do tonight.

This is a screenshot from a device which was working:

image

Then one from the device not working.

image

Nooooooo, screen shots are Noooooooo. Copynpasta the output - then we can a, read it and b, hack it about.

However I’d note that the channel (0x01) and the LPP_GPS aren’t present in the not-working received payload so the decoder, which is looking for LPP formated payload hasn’t got anything to get started on. Which explains the cause but not the symptoms.

u-blox aren’t that nasty they radically alter the output from their modules, but if your NEO 8M is an equivalent, there may be some difference. I’d get your working code base for your NEO 6M from the multiple layers of backup archive and just plug the NEO 8M in, just in case finger trouble made an alteration somewhere.

If not, then serial output as text, side by side, for both modules (hopefully you can swap them over in a jiffy), may reveal the issue. It will be made clearer if you can patch out the If newData test so we get to see what TinyGPS has provided, regardless.

GPS still works in the dark, even if you can’t see the satellites.

Thank you - sadly family life has intervened temporarily. I can get back on this tomorrow but I’m sure you are right, the data is not being identified as an LPP payload for some reason.

What I did manage (under the pretext of doing something else) was try the custom decode I used before I discovered Cayenne LPP did it for me. The result was confusing, it returned values but they were orders of magnitude out and changed massively between packets.

Not much help perhaps but another possible clue.

That is true, but also not 100% true… Newer devices have a floating point value in dgps_age of xxGGA where previously an integer was used.

For a typical GPS, there will be around 8 sentences a second and only two of them have new fix data.

So do not assume that as every NMEA sentence is received there is new (fix) data available.

This may appear to be of no importance, but using gps.encode() in that way can mean you dont get new fix data at all.

Loratracker: the code does work although if there is a more efficient solution I would be very interested. :slight_smile:

Kersing: Can you suggest how the code needs to be changed - even if only to see what might work?

Sorry but I don’t know the internals of the library you are using. Basically you need to check that code to see if the field I mentioned is handled as integer or float. Of course only if you need the contents of the message for your purpose, if you don’t need it it doesn’t matter if the parsing of that message fails.

This routine, for TinyGPS++ library, will wait a given number of seconds for an updated fix from the GPS. Returns true if the fix is updated, false if there is a timeout. The timeout is there so that the rest of your program is not hung if the GPS stops working for some reason. You can also get an updated fix when you want it, rather than having the GPS data being constantly read. Also useful for GPS hotfixing when the GPS is turned off most of the time to save power.

A running GPS, with a fix, will normally return from the gpsWaitFix() in under a second.

bool gpsWaitFix(uint16_t waitSecs)
    {
      //waits a specified number of seconds for a fix, returns true for updated fix

      uint32_t endwaitmS;
      uint8_t GPSchar;

      Serial.print(F("Wait GPS Fix "));
      Serial.print(waitSecs);
      Serial.println(F(" seconds"));

      endwaitmS = millis() + (waitSecs * 1000);

      while (millis() < endwaitmS)
      {
        if (GPSserial.available() > 0)
        {
          GPSchar = GPSserial.read();
          gps.encode(GPSchar);
          Serial.write(GPSchar);
        }

        if (gps.location.isUpdated() && gps.altitude.isUpdated() && gps.date.isUpdated())
        {
          return true;
        }
      }

      return false;
    }
1 Like

I’m doing some decorating today but have found time to run a couple of tests. As you suggested I loaded the new device with a sketch from a device which worked - and it was successful! I will do some more investigating but at least it seems the GPS is not the issue.

Many thanks!

I think I have discovered why it wasn’t working - I may have corrupted some of the keys. I created a new device and entered all the new keys into the sketch and it works. It was only working when attached to the PC at first and wouldn’t work on batteries. Then I noticed I hadn’t disabled frame counter checks and unchecking this seems to have got it working although it is still a bit erratic and returns several lines of zero coords in between a proper fix. There may be something wrong with a soldered joint so I will check it over physically. If I press the reset button on the Arduino it sends a valid location fix then reverts to sending zeros. When attached to the serial port it runs correctly. Very odd.

Edit to the above: There’s a bad joint somewhere - poking the wires gets it to work on batteries correctly. I will re-solder all the joints.

Well done!

I thought I’d got it working after finding a dry joint but it is still behaving erratically.

I’m going to have another look at it tomorrow. I might cut my losses and take it all apart and rebuild it. Been a long day of decorating!