Real-Time AI At The Edge May Require A New Network Solution

By Jim McGregor

AI has the power to change every electronic platform, but what works in the data center may not work in an industrial edge platform like a security camera, robotic arm, or even a vehicle. There is no one-size-fits-all for edge AI because of space, power, data security, and performance-latency requirements, which means there is not one solution for all AI applications. Transitioning to edge AI requires new solutions, especially for on-device training.

The Current AI Model

Most AI models start in the data center with the training of a neural network model based on public and/or private data. As we have argued in the past, the expense of running AI in the cloud can be prohibitive from a cost, latency, and security standpoint. As a result, neural network models can be optimized by shrinking the model size to then run at the edge of the network on the device, commonly referred to as “edge computing.” This optimization is a delicate balance of reducing the size of the model while maintaining acceptable accuracy.

While neural network models and optimization techniques improve, using a scaled-down data center solution may not be optimal for many edge applications. Edge applications are often focused on the input of sensor data and require even smaller models with a high degree of accuracy and real-time, or close to real-time, execution for mission critical or even life-threatening situations. These edge applications may include healthcare, automotive/transportation, manufacturing, and security.

Event-driven AI

As an alternative, BrainChip has developed its Akida neuromorphic IP (intellectual property) solutions to support Temporal Event-based Neural Networks (TENNs), an event-based neural processing network architecture, in addition to traditional Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNNs) and Transformer based neural networks. What this means is that a temporal (time) enabled neural network (TENN), or networks, is only operating during the time when a trigger event or input occurs. During other times, it is not computing and therefore not consuming much power. This can translate to higher performance, adaptability and lower real-time latency per event, input, or request and at a fraction of the power consumption of other AI solutions.

According to BrainChip, TENNs are ideal for processing various types of data such as one-dimensional time series and spatial-temporal data. TENNs is showing positive results in a number common applications, such as audio denoising, eye-tracking for AR/VR, health data monitoring (heart rate, SpO2), keyword spotting, Small Language Models (SLMs), and video object detection. TENNs ability to adapt to new input/events overcome some of the limitations of traditional neural networks while supporting future neural network models at a fraction of the die area and operating cost.