Lost records, lost commands, no device bug
Records missing from a download, and commands that stopped reaching trackers. Both looked like firmware bugs. One was an old reader tool, the other a carrier NAT that forgot devices after about 15 seconds of quiet.
When data goes missing between a tracker and a server, everyone looks at the device first. It’s the part we build.
These two tickets were both about lost data, and in both the device turned out to be fine. The work was proving where the data really stopped.
Missing records
The first ticket estimated that the “Read records” download in the configuration tool was missing about 30% of the records on the device. If true, that’s a firmware data-loss bug.
The desktop tool for opening the downloaded .bin file had been discontinued. So I wrote my own parser in Python. It decodes the device’s record protocol straight from the dump and counts every record.
On the file I checked, the old tool showed 101 of 201 records. My parser found all 201. The old tool just didn’t read them.
I published the parser in our internal tools docs, so anyone can check a download themselves. QA retested the download and it passed.
Missing commands
In the second ticket, trackers sent data to the server over UDP, and the commands the server sent back stopped arriving.
I reproduced it first. The modem was the obvious suspect, so I opened a case with the modem vendor. Then I suspected the forwarding on our office network. Neither explained it. The ticket stayed open from 2025 until I found the cause in February 2026.
A NAT with a short memory
On this operator’s APN, the device had no public address. It sat behind the operator’s carrier-grade NAT, which keeps a port mapping open only while traffic flows both ways. On this network, the mapping closed after about 15 seconds of quiet. Any command after that went to a port the NAT had forgotten, and was dropped before it reached the modem.
The fix was a setting: the existing network ping now runs every 10 s, so the NAT never sees 15 seconds of quiet. I also proposed a default ping interval for UDP links in the firmware, and updated the test automation script.
Conclusion
Records: all 201 were in a file the old tool read as 101. Commands: with a ping every 10 s, they reach the trackers again. The 201 is from the one file I checked, and 15 s is this operator’s timeout, not a universal number.
Neither fix touched firmware code. The NAT case took long because the first two suspects were reasonable ones.