TLS ate the RTOS buffer pool
A tracker rebooted every 20 s during the TLS handshake, but not on every boot. Every boot that crashed was running a vehicle-bus scan. TLS and the scan were drawing from the same RTOS buffer pool, so I gave TLS its own.
A tracker connecting to a cloud MQTT broker kept rebooting every 20 s, in the middle of the TLS handshake. But not every boot crashed.
That was the useful part. If some boots survive, the difference between them is the clue.
What the crashing boots had in common
I lined up each crash against everything else the device was doing. Boots 1 to 7 all crashed, and all of them were running a vehicle-bus scan. The one that survived had the ignition off, so no scan ran. And at every crash, all 97 of the larger buffers in the RTOS message-buffer pool were in use.
One pool, two users
The TLS library took its small memory blocks from the same RTOS pool that tasks use to message each other. One handshake took 42 of them. The vehicle-bus scan was using the pool too. Together they emptied it, and the RTOS treats that as fatal.
The fix
TLS now has its own private fixed-block memory, so it can’t drain the RTOS pool. Before connecting, the device also counts the free buffers. If there aren’t enough, it reports a resource error and tries again later instead of crashing. QA also got a way to create low-memory conditions on the bench.
Conclusion
The reboot loop stopped, and the customer confirmed that MQTT and vehicle-bus scanning now work together.
The crash happened in TLS, but TLS wasn’t really the problem. Sharing the pool was. When two things share a pool, the one that asks last is the one that crashes.