Multi-person localization with UWB radar on a microcontroller

This project is a TinyML system that estimates, frame by frame, the position of 0–4 people in a room from the raw signals of six UWB radars, with a model designed to run on an ESP32-S3 microcontroller. The int8-quantized network takes 90.3 KB, the complete pipeline reaches an F1 of 0.907 on the held-out validation windows, and the project received a 10/12 grade.

Context

The project was carried out for the Embedded and Edge Artificial Intelligence course at Politecnico di Milano (academic year 2025/26, September 2026 session), in a team with other colleagues. I wrote the code: preprocessing, model, training, decoding tuning, int8 quantization and export, and the inference script. The data analysis and the report were shared work.

The problem

Six TSRR250 UWB radars, mounted on the perimeter of a 4.8 × 7.2 m room, measure the channel impulse response (CIR): for each radar, 3 antennas and 120 range bins of 0.15 m, sampled at 25 Hz. For every frame, the system must return the list of (x, y) positions of the people in the room, from 0 to 4.

The target is an ESP32-S3 running TensorFlow Lite for Microcontrollers, so the model must stay under 800 KB of Flash and under 300 KB of SRAM for activations, with no recurrent layers (LSTM, GRU, RNN) and no custom operators. The inference script may only use NumPy, SciPy and the TFLite interpreter.

The main metric is F1 with Hungarian matching and a 1 m threshold: a prediction more than one metre from the true person counts as both a false positive and a false negative, while sub-metre precision earns nothing. So the pipeline gives up precision it does not need in exchange for the robustness it does need: to person-count errors, to a change of session and to int8 quantization.

From the data to the design

Before designing the model, we measured on the data the assumptions it would rest on.

• Range calibration. The nominal bin size (0.15 m) is consistent with the data, but each radar has an undocumented constant delay. Fitted on the four single-person windows, it ranges from +4.5 to +7.2 bins depending on the radar; with this correction, the energy contrast at the person's predicted bin rises from about 2.0× to 2.8–3.4×. A single global offset would have introduced up to 0.23 m of systematic error on two radars.
• Dataset structure. There are 24 acquisition windows of 300 s, and the number of people is constant within each one. Eight windows contain seated people, about 20% of all observations.
• Room borders. 0.26% of the true positions lie just outside the room, while the validator rejects out-of-bounds predictions: clipping predictions to the room is mandatory and costs at most 0.29 m on a few frames.

Preprocessing

The heaviest part of the work happens in NumPy, before the network.

• Clutter removal. The static environment sits 33–40 dB above the signal of moving people: a person is about 1.5% of the raw amplitude, and an int8 representation would crush it into a handful of levels. For each radar I subtract the temporal mean of the file, recomputed at inference time too, so the pipeline adapts to the test scene.
• Two channels. Clutter removal also deletes whoever stays still, because a person sitting motionless is the mean. So I compute two complementary channels: motion (magnitude of the clutter-free CIR, summed over antennas) and breathing (15 s RMS of the 0.1–0.6 Hz band, extracted with a zero-phase Butterworth filter). Instead of fusing them by hand, I pass them to the network as separate channels and let it learn a context-dependent fusion. The three antennas are summed: their magnitudes correlate at 0.84–0.88.
• Geometric backprojection. The room is divided into a grid of 24 × 36 cells of 0.2 m. A precomputed table maps every radar-cell pair to its calibrated range bin, so each cell reads the energy each radar sees at the right distance: the geometry is solved exactly, at no SRAM cost, instead of being learned from data. The six radars stay separate channels, because removing multilateration "ghosts" requires knowing which sensor contributed. The error from the cells (at most 0.14 m) is negligible against the 1 m threshold.
• Temporal context. Recurrent layers are not allowed, so each frame is stacked with frames t−5 and t−10 (0.4 s, past only). Consecutive frames are near-duplicates (median displacement of 0.018 m per frame), while spaced frames add information.

The network input is a 36 × 24 × 36 tensor (3 instants × 2 channels × 6 radars), normalized per frame, instant, channel and radar, so the absolute signal strength, which depends on session and hardware, does not matter.

The two channels fail in opposite situations. On two validation windows, this is the share of people found within 1 m by the strongest cells of a single channel, after backprojection:

Each channel alone caps recall; together they cover both cases. Fusing them by hand costs about 10 points on walking-only scenes.

The model

A small encoder-decoder CNN (77,857 parameters) turns the input into a 36 × 24 confidence heatmap: a stride-2 3 × 3 convolution halves the resolution straight away, followed by two convolutions with a residual connection, a 1 × 1 bottleneck, nearest-neighbour upsampling and a 1 × 1 sigmoid head. It uses only Conv2D, BatchNorm, ReLU, Add and UpSampling2D, all supported by TFLite Micro. The heatmap handles a variable number of people natively, unlike regression with fixed slots.

The architecture is shaped by the evaluator's arena estimate, which sums all activation tensors instead of taking the peak: every extra layer costs SRAM, so downsampling happens at the very first layer and the channels are thinned before upsampling.

Training minimizes a CenterNet-style focal loss on Gaussian targets (σ = 0.35 m), with two essential details:

1. the loss is normalized per frame, not per batch: 8% of the dataset is an empty room, and batch-level normalization inflates empty frames by about 30 times, destabilizing validation;
2. every batch draws from all windows at once, because batches from a single window give BatchNorm single-scene statistics that do not transfer.

Decoding

The heatmap is averaged over 9 frames (0.36 s: at the median speed of 0.45 m/s this blurs a person by about 0.16 m). Then I extract local maxima with non-maximum suppression of radius 0.45 m (below the minimum observed distance between two people, 0.73 m), apply a threshold, keep at most 4 detections and clip them to the room.

Threshold, radius and averaging are chosen by a grid search that maximizes F1 computed with the official evaluator's matching function, imported directly to rule out any reimplementation drift. Besides +0.014 F1, temporal averaging removes every false positive in the empty-room window, where single-frame peaks reach 0.586 and would otherwise cross the threshold.

Validation and results

A random frame-level split makes no sense here: consecutive frames are near-duplicates, and the final evaluation uses an unseen acquisition session. Validation therefore holds out whole windows, one per number of people, 5 of 24, including one with a seated person and one with the room empty.

These are the results of the complete pipeline (threshold 0.35, NMS 0.45 m, 9-frame averaging) on the validation windows:

In the fifth window, with the room empty, there are no false positives over 7,500 frames.

On the example provided by the course, derived from training data and useful only as a sanity check, the complete int8 pipeline reaches F1 0.983, RMSE 0.165 m and the correct person count in 93.5% of frames, in 8.7 s for 5,000 frames.

The ESP32-S3 constraints, checked with the course script:

Quantization is full int8, on weights and activations, calibrated on 256 representative frames taken from one window with walking people and one with seated people; the F1 loss against the float model is negligible. The model is small by design, not by compression: geometry and clutter removal live in the preprocessing, outside the network.

Limitations

• Entering and leaving. The number of people never changes within a training window, so people entering or leaving the room are never seen in training; temporal averaging only partly masks this.
• Early overfitting. The best validation loss comes at the second epoch: there is room for data augmentation (symmetries of the room and of the radar channels) that we did not exploit.
• Motionless people. The breathing channel assumes a breathing person: a fully static target would still be missed.
• Validation uncertainty. There is a single window per number of people, so the pooled F1 of 0.907 carries corresponding uncertainty.