SHF: Deep Neural Network Inference on Energy-Harvesting Devices

Intermittently powered, energy-harvesting computers are sophisticated computing, sensing, and communicating systems that do not need a battery or tethered power source. These energy-harvesting devices will form the foundation of the next generation of internet-of-things (IoT) applications, ranging from wearable and implantable medical devices, to environmental and atmospheric monitoring, to tiny ChipSat-scale satellites in deep space. Realizing the value of these applications requires intelligent devices that can frequently make decisions locally and autonomously (i.e., without help from other nearby computers). For example, a device may need to decide whether to turn on a battery-draining camera to detect a person of interest, to decide which sensors embedded in concrete to enable to collect the most useful data about an aging bridge, and to decide when and how much data to send from these sensors back to the cloud. In recent years, statistical inference and machine learning using deep neural networks has proven the most successful method for such decision-making. Machine learning is a crucially important feature for future IoT devices, but today’s resource-constrained energy-harvesting systems do not support the high-intensity computations required by deep neural network inference. This project builds the software computer systems and hardware computer architectures required by future, intermittent IoT devices to enable autonomous, intelligent decision-making using machine learning. This project produces software systems with novel algorithms that enable today’s energy-harvesting IoT devices to efficiently make intelligent decisions. This project will then design novel parallel computer architectures that are designed specifically for efficient operation of machine learning computations with intermittent input power. These architectures further increase the efficiency of intermittent decision-making by 10s or 100s of times, enabling a new class of intelligent IoT applications that are not possible using today’s architectures. The sum of these software and hardware components addresses the existential question of deep machine learning on intermittent systems, demonstrating its viability and realizing its benefits to academia, industry, and in applications important to society, such as defense, healthcare, and civil infrastructure. This project contributes towards a diverse future workforce, through integration with course curricula, mentoring of students from under-represented minority groups, and technical high school outreach programs.

The key challenge overcome by this project is to make deep neural network inference viable on a resource-constrained, intermittent device. This task requires architecture and software support to tolerate frequent, intermittent power interruptions and to operate with hundreds of microwatts of power instead of the tens or hundreds of milliwatts required by today’s machine learning accelerators. This project develops approximate, intermittent partial re-execution techniques to efficiently tolerate interruptions without the need to unnecessarily checkpoint and restore software state. The project develops the first intermittence-safe data-parallel architecture, integrating non-volatile memory with an array of simple compute elements. The project includes an immediate path to software and hardware prototypes and lays the groundwork for a future silicon hardware implementation of the architecture.

This is a joint award with Brandon Lucia. See NSF award CCF-1815882.

Sponsor

National Science Foundation

Projects

Energy-Minimal Programmable Architectures for Intelligence Beyond the Edge

Publications

Pipestitch: An energy-minimal dataflow architecture with lightweight threads [pdf]

Nathan Serafin, Souradip Ghosh, Harsh Desai, Nathan Beckmann, Brandon Lucia. MICRO 2023.

Dataflow Blocks: Modular Time-Multiplexing for CGRAs [pdf]

Xuesi Chen, Nishanth Subramanian, Karthik Ramanathan, Nathan Beckmann, Brandon Lucia. YArch at ASPLOS 2023.

MANIC: A 19µW @ 4MHz, 256 MOPS/mW, RISC-V Microcontroller with Embedded MRAM Main Memory and Vector-Dataflow Co-Processor in 22nm Bulk FinFET CMOS [pdf]

Graham Gobieski, Oguz Atli, Cagri Erbagci, Ken Mai, Nathan Beckmann, Brandon Lucia. ISCAS 2023.

RipTide: A programmable, energy-minimal dataflow compiler and architecture [pdf]

Graham Gobieski, Souradip Ghosh, Marijn Heule, Todd Mowry, Tony Nowatzki, Nathan Beckmann, Brandon Lucia. MICRO 2022.

SNAFU: An Ultra-Low-Power, Energy-Minimal CGRA-Generation Framework and Architecture [pdf]

Graham Gobieski, Oguz Atli, Ken Mai, Brandon Lucia, Nathan Beckmann. ISCA 2021.

MANIC: An Energy-Efficient Architecture for Ultra-Low-Power Embedded Systems [pdf]

Graham Gobieski, Amolak Nagi, Nathan Serafin, Mehmet Meric Isgenc, Nathan Beckmann, Brandon Lucia. MICRO 2019.

Intelligence Beyond the Edge: Inference on Intermittent Embedded Systems [pdf]

Graham Gobieski, Brandon Lucia, Nathan Beckmann. ASPLOS 2019.

Intermittent Deep Neural Network Inference [pdf]

Graham Gobieski, Nathan Beckmann, Brandon Lucia. SysML 2018.