A few years ago, I was asked to perform the audit of an embedded device. This device was used in professional fleet management applications, to upload various data to a central application server: vehicle location and speed, logic states of a few digital inputs, etc. The technical archictecture was based on a microcontroller and a GSM/GPRS/GPS module. An over-the-air (OTA) update function had been implemented, so that remote software update was possible. Nothing really complex, nor revolutionary.
Several thousand vehicles had been equipped. But quite rapidly, up to one third of installed devices stopped working. And the most stressful point was that the OTA update was no more functional. Consequently, every non-functioning device had to be removed from the vehicle, reprogrammed, and reinstalled.