One SMS, 100 replies, and a reboot loop
One SMS command got over 100 replies. One modem event put trackers in a reboot loop. Both times the modem said something the firmware didn’t expect. I capped the resends, taught the parser the new format, and five blocked tickets were cleared.
The firmware talks to the cellular modem through AT commands. It sends a command, and the modem answers or reports events on its own. Every handler in the firmware assumes something about what the modem will say.
Here are two times, a year apart, when the modem said something else.
100 replies to one SMS
In 2022 a user sent one SMS command to a tracker and got more than 100 replies.
Two things together caused it. The firmware resends an SMS reply if sending fails. And this modem returned an error for SMS sends even when the message had gone out. So each reply reached the user, the firmware thought it had failed, and sent it again.
The firmware can’t tell a fake error from a real one, so there’s no clean fix on our side. What I added is a safety net: once 10 replies are queued, the firmware restarts the modem instead of resending forever.
A reboot loop from three words
In 2023, trackers with one of our external modems went into an endless restart loop, with the default configuration. Five other tickets were blocked behind this one.
The modem reports packet data events with +CGEV. The handler for it sits in about 3,000 lines of URC handling in C. This modem also sent three-word events like +CGEV: ME PDN ACT. The handler had no case for that, parsed it wrong, and the device fell into a restart loop.
I added the three-word cases, ME PDN and NW PDN, so they’re parsed in their own format. The fix was merged to the release branch the same week.
Back to the same event, later
I came back to +CGEV more than once. In 2024 I used +CGEV: ME DETACH in traces to explain why a link closed right after a server command, and showed the flow was the same on every modem. In 2025 I changed how another modem family handles a network detach: the firmware now asks the modem for the context’s current IP, and keeps the connection open if the modem has already recovered.
Conclusion
The restarts stopped, and all five blocked tickets were cleared. The SMS cap stops the loop, but it doesn’t fix the modem’s false error.
Both bugs come from the same habit: trusting whatever the modem says. A modem handler has to cope with answers that are wrong, and with answers it has never seen.