Neural Decision Processors (NDP)

While the TML120 provides the physical package, the NDP is the specialized engine driving the inference. The heart of the eFabric™ ecosystem is the Neural Decision Processor (NDP). This is a category-defining piece of silicon, engineered from the transistor level up to do one thing: execute deep neural networks with maximum efficiency. Unlike a general-purpose processor that treats AI as just another "task" or "program," the NDP is designed so that the architecture of the silicon itself mirrors the architecture of a neural network.

---

A. Silicon-Native Intelligence

In the eFabric™ workflow, we don't write "code" to run your model. Instead, the platform maps the neural network’s synapses and neurons directly onto the hard-wired logic of the NDP.

• The "Baked-In" Advantage: Because the logic is physical rather than virtual (software), there is no "instruction fetch" overhead.
• Deterministic Performance: The time it takes for a signal to move from a sensor input to a neural decision is near-instantaneous and perfectly predictable, which is critical for real-time safety and trigger applications.

---

B. The At-Memory Power Advantage

The defining characteristic of the NDP is its At-Memory compute architecture. In traditional systems, energy is wasted moving data back and forth across a "bus." The NDP eliminates this "energy tax" by storing the weight parameters directly within the computational gates. This results in a massive leap in operational efficiency ()

$$\eta_{NDP} \approx 1000 \times \eta_{MCU}$$

This allows the processor to remain in a state of "Always-On" monitoring while consuming only microwatts—power levels so low they are often less than the natural battery leakage.

The Gatekeeper Philosophy tip

"The NDP isn't just a processor; it's a filter. It sits at the edge of your system, silently processing streams of data. It only alerts the rest of the device when it 'hears' or 'feels' something important, acting as the ultimate gatekeeper for system energy."

Dedicated ML Co-Processors vs. General-Purpose MCUs

In the early days of Edge AI, developers were forced to use general-purpose Microcontrollers (MCUs)—such as the ARM Cortex-M series—to run neural networks. While these MCUs are versatile, they are "Jacks of all trades, masters of none." To achieve the Microwatt Scale required for eFabric™ applications, we must replace this "Software-Defined" approach with Hardware-Defined execution via the Neural Decision Processor (NDP).

A. The Computational Paradigm: Instructions vs. Circuits The fundamental difference lies in how the processor treats a mathematical operation like a Multiply-Accumulate (MAC), which is the heartbeat of every neural network.

General-Purpose MCUs (Software-Defined): An MCU is an Instruction-Set Architecture (ISA). To solve a single neural layer, it must fetch an instruction, decode it, fetch the data from RAM, perform the math in the ALU and write it back. This cycle repeats millions of times, consuming significant energy at every "hop" between the CPU and memory.

The NDP (Hardware-Defined): The NDP is a Data-Flow Architecture. There are no "instructions" to fetch. The neural network's layers are mapped directly onto the physical silicon gates. When data flows in, the math happens as a concurrent physical reaction through the circuitry.

B. The Energy Gap: Breaking the 1mW Barrier In a traditional MCU, the "energy tax" is high because the CPU and the system bus must remain powered up to move data.

• The MCU "Tax": Even at low clock speeds, a Cortex-M4 might draw 5mA to 10mA while running an AI model. On a standard 220mAh CR2032 battery, this would last only a few days.

• The NDP Advantage: By eliminating the memory bus and the CPU overhead, the NDP performs the same inference while drawing less than 150µA (microamps). This is the difference between a device that needs charging every week and one that lasts for three years on a single cell.

Feature	General-Purpose MCU (Cortex-M)	Syntiant® NDP (Dedicated Silicon)
Logic Type	Programmable Logic (Software)	Fixed-Function Neural Logic
Data Movement	CPU <-> RAM (High Energy)	At-Memory (Near-Zero Energy)
Parallelism	Sequential (One op at a time)	Massive (Matrix-wide ops)
Always-On Feasibility	Poor (Drains battery rapidly)	Native (Designed for Always-On)

C. Determinism: Reliability in the Millisecond

Because an MCU is often running an Operating System (RTOS) and handling other tasks like Bluetooth or Wi-Fi, its response time to a sensor event can vary. This is Jitter.

The NDP is a Deterministic Processor. Because it is hard-wired for the model, the time from "Sensor Input" to "Neural Decision" is always constant. In mission-critical applications—like detecting a glass break or a high-pressure valve failure—this millisecond-level consistency is vital.

Technical Note: The Gatekeeper Strategy

"Don't view the NDP as a replacement for your MCU, but as its 'Security Guard.' The NDP stays awake at the Microwatt level, filtering out 99.9% of the noise. It only 'wakes up' the power-hungry MCU when a high-confidence event is detected, preserving the system's energy for high-level logic.

D. The Mathematics of Efficiency

For quantification, we use the Energy per Inference (EI) metric:

$E_I = P_{\text{avg}} \times t_{\text{inference}}$

In an MCU, Pavg (Average Power) is high due to the peripheral overhead. In an NDP, Pavg is minimized because only the necessary neural gates are toggled.

---

The Syntiant NDP Silicon Family Architecture

The eFabric™ ecosystem leverages a suite of specialized Neural Decision Processors (NDPs), each tailored for specific data types and power budgets. While all members of the family share the same "At-Memory" DNA, the architecture evolves from simple "Keyword Spotters" to multi-modal engines capable of complex vision and vibration analysis.

A. The Evolutionary Hierarchy

The NDP family is categorized by its computational capacity and the complexity of the neural networks it can support.

Series	Focus	Key Feature	Power
NDP10x	Audio Sentinels	Fully Connected Networks (FCN)	< 100µW
NDP12x	Multi-Modal	CNN & RNN Support (Audio/IMU)	< 1 mW
NDP200	Edge Vision	Image/Video & Complex Sensor Fusion	< 5 mW

• NDP10x Series (The Ultra-Low-Power Sentinel): Designed primarily for "Always-On" audio. These chips focus on Fully Connected Networks and are optimized for detecting specific keywords or simple acoustic events (like a glass break) while consuming under 100µW.

• NDP12x Series (The Multi-Modal Workhorse): The core of the TML120 module. This architecture introduces support for CNNs (Convolutional Neural Networks) and RNNs (Recurrent Neural Networks). It can process audio, pressure, and IMU (motion) data simultaneously, making it the standard for sophisticated wearables and industrial sensors.

• NDP200 Series (The Edge Vision Specialist): The powerhouse of the family. It features a larger neural engine capable of processing low-resolution image data for person detection or Time Series Model, alongside advanced audio and vibration analysis.

B. Internal Architecture: The Three Pillars

Regardless of the model, every Syntiant NDP is built on three architectural pillars:

1. The Neural Engine: This is the matrix of silicon gates where the model resides. It is optimized for Int8 quantization, allowing the chip to perform millions of 8-bit math operations in parallel without the overhead of a floating-point unit.

2. The Integrated DSP: Before data hits the neural engine, it passes through an on-chip Digital Signal Processor. This block handles the "Dirty Work"—FFTs, Mel-binning, and filtering—so the neural engine only sees "clean" features.

3. The Management Core: A tiny, low-power controller that handles the "handshake" with the outside world (like SPI or I2C communication). It ensures the chip stays in a low-power state until a detection is made.

C. Scaling for the Application

Choosing the right architecture is a balance of Memory vs. Intelligence.

Audio Applications: Focus on the NDP120 for its ability to handle background noise suppression (beamforming) alongside keyword detection. Engineers must calculate if their model parameters (P) fit the silicon’s memory capacity using the required memory formula (Mreq):

$$M_{req} = \sum_{i=1}^{L} \left( P_i \times \text{bits\_per\_weight} \right)$$

Industrial Vibration: Utilizes the high-speed SPI interfaces of the NDP120 to ingest high-frequency accelerometer data for real-time anomaly detection.

📊Data Insight: The "Resolution" Factor

"When selecting an NDP, consider the 'Feature Resolution.' While an NDP101 can recognize a single wake-word with 5 spectral bins, the NDP120 can process 64 Mel-bins, allowing it to distinguish between the sound of a 'refrigerator compressor' and a 'failing industrial motor' with much higher precision."

---

Memory Constraints and Neural Network Compression

In traditional computing, if a model is too large, you simply add more RAM or swap data to a hard drive. In the At-Memory world of the NDP, memory is not a flexible resource—it is a physical part of the processor's fabric. This creates a hard ceiling: your model must fit within the silicon's fixed weight storage, or it cannot run. Compression is the engineering discipline of "shrinking the brain" without losing the "intelligence."

A. The Hard Reality of "At-Memory" Limits Because the NDP stores weights directly within the computational gates to save energy, the memory is static.

• The Constraint: For a typical edge device, you might have between 64KB and 1MB of space for the entire model.

• The Challenge: A standard "research" model (like a ResNet or a large Transformer) can be hundreds of megabytes. Deploying on eFabric™ requires a reduction in size of up to 1000x.

B. The Three Pillars of Compression To squeeze maximum intelligence into these tiny silicon footprints, eFabric™ employs three primary mathematical strategies:

1. Quantization (Precision Reduction)

Standard AI models use 32-bit floating-point numbers (FP32). The NDP is optimized for Int8 Quantization.

• The Math: We map the wide range of a 32-bit decimal into a narrow range of 8-bit integers (0–255 or -128 to 127).

• The Result: This immediately reduces the memory footprint by 75% (4 bytes reduced to 1 byte) with minimal loss in accuracy.

2. Weight Pruning (Sparsity)

Not every connection in a neural network is important. Many weights are so close to zero that they contribute nothing to the final decision.

• The Process: eFabric™ identifies these "lazy" neurons and removes them.
• The Benefit: By "thinning out" the network, we free up physical gates for more critical features. A pruned model is faster and smaller but retains the same "decision-making power."

3. Knowledge Distillation (Teacher-Student)

This is the ultimate compression hack. We train a massive "Teacher" model on a powerful GPU and then "teach" a tiny "Student" model (the one destined for the NDP) to mimic the teacher’s behavior.

• The Goal: The student model learns the shortcuts and patterns of the larger model without needing the massive overhead.

C. Calculating the Footprint (Mtotal)

Engineers must ensure the model remains under the Hardware Budget. The total memory The total memory required (Mtotal) and the success of pruning (Rp) are calculated as follows:

$$M_{total} = \sum_{i=1}^{L} \left( P_i \times S_{quant} \right)$$

• $L$: Number of layers
• $P_i$: Number of parameters in layer $i$
• $S_{quant}$: Size of quantized weight (in bits) and

$$R_p = 1 - \frac{\text{Weights}_{pruned}}{\text{Weights}_{original}}$$

📊Data Insight: The Accuracy-Memory Trade-off

"In the eFabric™ Factory, you will see a 'Compression vs. Accuracy' curve. The goal isn't the smallest model possible; it’s the smallest model that still meets your target Precision/Recal metrics before accuracy 'falls off a cliff'."

---

💡Memory Tip

"Always start your design with Quantization-Aware Training (QAT) in eFabric™. It allows the model to 'learn' to be 8-bit during training, which results in much higher final accuracy than converting a 32-bit model after the fact."