Skip to main content

Pre-processing and Transformation

Think of Pre-processing as the "translator" that sits between the chaotic real world and the precise logic of the neural network. In a cloud environment, you might have the luxury of feeding raw data into a massive GPU and letting it "figure it out." At the edge—especially at the Microwatt Scale—this is impossible.

In the eFabric™ workflow, raw data is never fed directly into a neural network. Raw time-domain signals (like audio waves or accelerometer streams) are too dense and "noisy" for ultra-low-power silicon to process efficiently.

Pre-processing is the critical engineering stage where we transform raw signals into a compressed, structured format called Features. For the TML120 module, this transformation is handled by specialized Digital Signal Processing (DSP) blocks that ensure the "Intelligence" is delivered to the neural gates with zero wasted energy.

Here is a deeper look at why this stage is the "Secret Sauce" of the eFabric™ efficiency.

The "Why": The Efficiency Gap​

Raw data is essentially a "firehose." If you are sampling audio at 16,000 samples per second, that is 16,000 individual data points the neural network would have to process every second.

  • The Problem: Processing raw samples requires massive amounts of memory and trillions of operations (MACs), which would drain a coin-cell battery in minutes.

  • The eFabric™ Solution: We use DSP (Digital Signal Processing) to compress those 16,000 points into a handful of "Features"—mathematical summaries that retain the intelligence while discarding 99% of the useless data.

The "How": Transformation into Features​

In the eFabric™ workflow, we shift from the Time Domain to the Frequency Domain.

1. Dimensionality Reduction​

Instead of looking at the "wiggle" of a sound wave (Time Domain), the DSP blocks look at the "ingredients" of the sound (Frequency Domain).

  • Analogy: It’s the difference between describing a cake by listing every molecule (Raw Data) versus listing the ingredients: flour, sugar, eggs (Features). The neural network is much faster at recognizing "Cake" if it only has to look at the ingredients.

2. Feature Extraction Pipeline​

This is handled by the specialized hardware on the NDP silicon:

  • Filtering: Removes the "hum" of electronics or the "rumble" of gravity.
  • Spectral Analysis: Breaks the signal into frequency bands. For audio, we use the Mel-Scale, which groups frequencies the way a human ear does.
  • Normalization: Ensures that if a user whispers or shouts, the "Feature" looks the same to the neural network.

The Hardware Edge: Silicon-Native DSP​

The reason we mention "zero wasted energy" is because of where this math happens. In most systems, the main CPU has to wake up to do this math. In eFabric™:

  • The NDP (Neural Decision Processor) has a dedicated, hard-wired DSP engine.
  • This engine performs the math (FFTs, Log-scaling) using specialized circuits that are thousands of times more efficient than a general-purpose processor.
  • This means the "Intelligence" (the features) is delivered to the neural gates without ever "waking up" the rest of the system.
DSP-Neural Co-Design

"In eFabric™, the DSP settings and the Neural Network architecture are linked. If you change your 'Window Size' in pre-processing, you are essentially changing the 'resolution' of the data the model sees. Always re-validate your model after adjusting DSP parameters.”

How this benefits the Developer:​

  • Lower Latency: Features are smaller, so the neural network makes decisions faster.
  • Smaller Models: Because the data is "cleaner," you need fewer neural layers to reach high accuracy.
  • Ultimate Battery Life: This is how we achieve Always-On capabilities that last for years on a single battery.

The Mathematics of Feature Extraction​

To achieve Silicon-Native Execution at the microwatt scale, we must move beyond the "noisy" time-domain signal. This section details the mathematical sequence used to convert raw sensor oscillations into a 2D "image" of features that the neural network can interpret with high precision.

A. Framing and Windowing (Temporal Segmentation)​

A raw signal is a continuous stream of amplitudes. Because its statistical properties change over time (non-stationary), we must analyze it in small, discrete "frames."

  • The Frame (Segmentation): We typically slice the signal into frames of 20ms to 40ms. This duration is the "sweet spot"—long enough to capture a meaningful frequency pattern, but short enough to be considered "stationary."

  • The Hamming Window: If we simply cut the signal, we create "sharp edges" that result in mathematical artifacts called Spectral Leakage. To prevent this, eFabric™ applies a Hamming Window function to each frame:

    w[n]=0.54−0.46cos⁡(2πnN−1)w[n] = 0.54 - 0.46 \cos\left(\frac{2\pi n}{N-1}\right)

    This function "tapers" the signal at the edges, ensuring the beginning and end of each frame smoothly transition to zero.

B. The Fourier Transformation (Time to Frequency)​

Once the frame is windowed, it is passed through a Fast Fourier Transform (FFT). This is the "Engine of Discovery" in eFabric™.

  • The Conversion: The FFT decomposes the complex time-domain wave into its constituent frequencies. It identifies how much "energy" is present at every frequency (Hertz) within that frame.
  • **Power Spectrum:**We calculate the Periodogram estimate of the power spectrum:
Pxx(f)=1N∣∑n=0N−1x[n]e−j2πfn∣2P_{xx}(f) = \frac{1}{N} \left| \sum_{n=0}^{N-1} x[n] e^{-j 2 \pi f n} \right|^2

This provides the raw spectral data, but it is still too high-resolution (too many data points) for a tiny chip.

C. Mel-Filterbank Processing (Bio-Inspired Compression)​

Human hearing and mechanical vibrations don't follow a linear scale. We are much better at detecting changes at 500Hz than at 15,000Hz. eFabric™ uses this principle to compress data via the Mel Scale.

  • The Mel Filterbank: We pass the FFT output through a series of triangular filters (typically 32 to 64 "bins"). These filters are spaced linearly at low frequencies and logarithmically at high frequencies.
  • The Formula: The conversion from Frequency (f) to Mel (m) is:
m=2595log⁡10(1+f700)m = 2595 \log_{10} \left(1 + \frac{f}{700} \right)
  • The Result: This process reduces thousands of FFT points into a small, dense vector of "Mel-bins," drastically lowering the memory footprint while retaining all the intelligence needed to identify a "Wake Word" or "Bearing Failure."

D. Log-Scaling (Dynamic Range Normalization)​

The final mathematical step is applying a Logarithmic Scale to the energy in each Mel-bin.

  • Human Perception: Humans perceive sound intensity logarithmically. By taking the log of the energy, we ensure the "Pattern" of the signal remains the same regardless of the volume.
  • Model Robustness: This step ensures the neural network identifies the relative relationship between frequencies rather than the absolute amplitude. This is why eFabric™ models work just as well when a user is 5 meters away as when they are right next to the device.
📊 Data Insight: The Nyquist Connection

"Your FFT resolution is directly tied to your sampling rate. With the eFabric™ default of 16kHz, your frequency bins will span from 0Hz to 8000Hz. If your target event (like a high-speed turbine) produces signatures above 8kHz, the Mel-bins in this section will be empty unless you increase the hardware sample rate."

Signal Processing for Sensors (IMU)​

While audio pre-processing focuses on the frequency domain, IMU (Inertial Measurement Unit) data requires a combination of time-domain and statistical filtering. For the TML120 module, the goal is to transform raw accelerometer (x, y, z) and gyroscope data into a "motion fingerprint" that is invariant to gravity and sensor noise.

A. DC Offset and Gravity Removal​

The most significant challenge with accelerometers is the constant 1g (9.8 m/s2) acceleration of gravity. If a device is tilted, gravity "bleeds" into different axes, creating a massive DC offset that can mask the subtle vibrations or gestures you are trying to detect.

  • High-Pass Filtering: eFabric™ uses a First-Order High-Pass Filter (HPF) to remove this constant bias. This ensures the model only "sees" the dynamic movement (the change in motion) rather than the orientation of the device. The Difference Equation:

    y[n]=α(y[n−1]+x[n]−x[n−1])y[n] = \alpha (y[n-1] + x[n] - x[n-1])

B. Signal Smoothing and Jitter Reduction​

Raw sensors are sensitive to electrical interference and micro-vibrations (jitter) that are irrelevant to human gestures or industrial anomalies.

  • Low-Pass Filtering (LPF): To clean the signal, we apply an LPF. This "smooths" the waveform, ensuring the neural network focuses on the Low-Frequency components typical of human movement (<20Hz) or mechanical shifts.
  • The Moving Average: Often, a simple moving average is used to stabilize the signal:
y[n]=1M∑k=0M−1x[n−k]y[n] = \frac{1}{M} \sum_{k=0}^{M-1} x[n-k]

C. Feature Extraction: Statistical vs. Spectral​

For IMU data, eFabric™ allows you to choose between two types of "Motion Features":

  1. Statistical Features (Time-Domain): Instead of every data point, we feed the model summaries of the window:
    • Mean & Standard Deviation: Captures the intensity of the movement.
    • Zero-Crossing Rate (ZCR): Measures how often the signal changes direction—a key indicator of vibration frequency.
    • Signal Magnitude Area (SMA): Useful for distinguishing between activity and rest.
  2. Spectral Features (Frequency-Domain): For applications like Predictive Maintenance, we apply the FFT (as we do in Audio) to the vibration data. This allows the model to detect the specific "whine" of a motor or the "shudder" of a failing bearing.

D. Vector Magnitude Normalization​

To make a gesture model "Orientation Independent," we often calculate the Euclidean Norm(Magnitude) of the three axes. This allows the model to detect a "shaking" motion regardless of whether the device is held vertically or horizontally. The Magnitude Formula:

∥a∥=x2+y2+z2\|a\| = \sqrt{x^2 + y^2 + z^2}
⚠️ Technical Note: The Nyquist Limit in Sensors

"While audio uses 16kHz, most sensor applications run at much lower sampling rates (e.g., 50Hz for human activity, 1600Hz for industrial vibration). Ensure your Sampling Frequency (fs) is at least double the frequency of the movement you wish to detect to avoid aliasing in your motion signatures."

In the world of TinyML, Data Augmentation is your most powerful tool for "teaching" a small model to handle a large, complex world. Because we are constrained by the memory of the TML120 module, we can’t just make the model bigger to handle noise; we have to make the model smarter by exposing it to every possible variation of the signal during the training phase.

Data Augmentation Strategies​

Data Augmentation is the process of mathematically generating new training samples from your existing dataset. By "stress-testing" your samples in the eFabric™ Factory, you ensure that the final model is robust enough to perform in the field with near-zero False Rejections.

A. Time-Domain Augmentation (Audio & Sensors)​

These techniques modify the signal's timing and intensity without altering its fundamental frequency components.

  • Time-Shifting: We randomly shift the event (keyword or gesture) forward or backward within the processing window. This ensures the model isn't "position-dependent"—it will trigger whether the keyword starts at 0ms or 400ms into the buffer.
  • Gain & Volume Scaling: We multiply the amplitude of the signal by a random factor (0.5 to 2.0). This simulates a user speaking from different distances or a sensor being mounted in different orientations.
  • Time-Stretching: Slightly speeding up or slowing down the signal (+/- 10%). This helps the model recognize the same keyword spoken quickly in excitement or slowly in a calm state.

B. Environmental Injection (The Noise Profile)​

The "secret" to eFabric™ robustness is Additive Noise Injection. Instead of training on "clean" data, we mathematically blend your signal (s) with a noise profile (n). The Mixing Formula:

xaug(t)=(1−λ)s(t)+λn(t)x_{\text{aug}}(t) = (1 - \lambda)s(t) + \lambda n(t)
  • α (The Mixing Coefficient): This determines the Signal-to-Noise Ratio (SNR).
  • eFabric™ Advantage: You can upload specific "Background Profiles" (e.g., a bustling cafe, a windy street, or a humming data center). The platform then creates thousands of variations of your keyword "living" inside those environments.

C. SpecAugment: Frequency-Domain Masking​

Developed for high-performance speech recognition, SpecAugment is a "pixel-level" augmentation performed directly on the Log-Mel Spectrogram.

  • Frequency Masking: We "black out" a random horizontal strip of the spectrogram. This forces the neural network to learn the other frequencies of the sound, so it can still identify the keyword even if a specific frequency is blocked by a loud whistle or fan.
  • Time Masking: We "black out" a random vertical strip (a slice of time). This teaches the model to handle "Drop-outs" or sudden interruptions in the audio stream.

D. Sensor-Specific Augmentation (IMU)​

For motion data, we use physical transformations:

  • Jitter Injection: Adding Gaussian noise to simulate electrical interference in the sensor wiring.
  • Axis Swapping: Randomly swapping the X, Y, and Z axes (if the movement is theoretically omnidirectional) to make the model truly orientation-independent.
📊 Data Insight: The Augmentation 'Sweet Spot'

"While augmentation is powerful, over-augmentation can 'drown out' the signal. If your α (noise level) is too high, the model will fail to converge because it can no longer see the pattern through the static. Always validate your model against Clean Data first to ensure it has actually learned the core signature."