Bringing AI to Embedded Devices via Neural Architecture Search

There is a rising amount of data produced in the world due to technologies surrounding Industry 4.0, Smart X, IoT, etc. This also increases the demand for efficient processing of data to reduce the costs for energy and infrastructure. An interesting trend is that processing is moving towards the edge, where the data originates from. In the extreme, data is being processed on the device that generates the data to avoid sending meaningless information. 

Getemed develops and produces medical devices, including wearable long-term ECG recorders, that can detect cardiac arrythmia on the device and send warnings to the doctor. Together with Fraunhofer ITWM and RPTU Kaiserslautern, the goal is to bring AI into the device, so it can operate more energy efficient, without losing precision. This allows a higher wearing time of the device before a change of battery is needed, which increases the diagnostic success. The challenge is to fit the AI application onto a device, that operates in the single digit milliwatt range. 

There are two points that helped us to succeed with this difficult task: We reconsidered the hardware and used an FPGA chip instead of a microprocessor for computation and we implemented deep neural networks (DNN) as the AI algorithm. We found that hardware-aware neural architecture search (NAS) is the key to enable AI applications on embedded devices. The NAS automatizes the DNN design with consideration for the hardware implementation. This approach strongly boosted our productivity and allowed us to quickly find a solution that meets our requirements. We believe this approach scales to different applications as well. In this presentation we give key insights on our experience with hardware-aware NAS and how we think it can benefit also other applications.