robotics · active
DR-CVaR — reproducing a distributionally-robust safety filter, honestly
A clean-room reproduction of my lab's ICRA'24 distributionally-robust CVaR safety filter, moved off the paper's idealized plant onto a car-like platform — and the verification story of what actually survives the move.
DR-CVaR is a robustness margin on a committed maneuver, not the thing that makes the pass possible — and the real guarantee lives in the closed loop that keeps re-establishing it, not in the safe reference you compute once.
Work in progressLive, but still being revised — the writing, diagrams, and data figures are all actively being critiqued and improved.
- arXiv:2309.08821Paper reproduced
- CVaR 29/30 → DR-CVaR 12/30Distribution shift, over-bound rate
- 7Verification-bug instances found + fixed
- oriented footprint, not discHonest collision predicate
- 128Tests green
- hardware (VT5)Next rung
The paper, and why reproduce it
A safety filter sits between a controller and the plant: the controller proposes a motion, the filter minimally reshapes it to stay safe, and the reshaped command is what drives. The appeal is separation — you can develop an aggressive planner and bolt a safety guarantee on afterward, rather than baking caution into every controller.
The filter I reproduced is my lab’s own: Distributionally Robust CVaR-Based Safety Filtering (Safaoui & Summers, ICRA 2024, arXiv:2309.08821). Its idea is to be robust not just to an obstacle’s predicted uncertainty, but to the prediction being the wrong distribution. Instead of trusting a single Gaussian forecast of where the obstacle will be, it solves for a separating halfspace that keeps the Conditional Value-at-Risk of a collision under a bound for every distribution within a Wasserstein ball around the forecast. You pay for the distribution you didn’t model.
I built the original version of this filter at CONLab during my bachelor’s. Rebuilding it clean-room — ROS-free, on the conlab v2 stack — was a chance to do something more useful than re-derive the math: to move it off the paper’s idealized plant and find out what the guarantee is actually worth on a car.
The honest headline — a margin guard, not a pass-maker
The paper’s filter is validated on a 10 g holonomic double integrator — a point mass that can
accelerate in any direction, hard. A COMO RC car is a car-like bicycle running ~1.25 m/s with a ~0.5 g
acceleration box and a steering constraint. That gap is not cosmetic. Under a real actuator box, a
head-on centerline reference has no lateral escape and simply collides — the filter, on its own,
cannot invent an in-lane overtake from a head-on reference. So the honest scenario is a guarded
maneuver: the ego commits an explicit FOLLOW → OVERTAKE → MERGE state, and DR-CVaR guards that
committed bow against prediction uncertainty.

The whole filter in one frame, drawn on a real run at the decisive instant. Orange is the DR-CVaR separating halfspace and its safe side; dashed purple is the plain-CVaR separator it would have used with no distributional robustness. The gap between them — a constant 0.25 m here — is exactly what the Wasserstein ball buys. The pass clears by 0.29 m on the oriented-footprint gap between the two 0.60 × 0.30 m bodies.
This reframes the result rather than weakening it. On the honest footprint predicate (below) the raw committed maneuver already clears the obstacle by +0.148 m; the DR-CVaR guard roughly doubles the clearance to +0.339 m. So DR-CVaR’s contribution on a realistic platform is a robustness margin on a maneuver the ego commits — not the thing that makes the pass possible. Saying that plainly, with the number, is the contribution: the paper’s headline was measured on a plant no RC car can drive, and this reproduction shows exactly what survives on one that can.
What distributional robustness actually buys
The clearest positive result is a distribution-shift stress test. Each separator is designed against a Gaussian obstacle cloud, then the obstacle is realized under a heavier-tailed Laplace at the same centre and scale — a deliberate mismatch between the modelled and the true distribution. The plain-CVaR separator, which trusts its Gaussian, breaches its own risk bound almost every time. The DR-CVaR separator, hedged over a Wasserstein ball, breaches it far less.

Robustness to the wrong distribution, across 30 seeds. Designed on a Gaussian, realized under a Laplace tail: plain CVaR exceeds its risk bound 29/30 times, DR-CVaR only 12/30. This is the property the whole method exists to buy, and it reproduces cleanly.
Underneath it is a cleaner invariant that also reproduces: the conservatism ordering Mean ≤ CVaR ≤ DR-CVaR in the halfspace offset, monotone in the Wasserstein radius. DR-CVaR is always at least as cautious as CVaR, which is at least as cautious as the mean separator.
Where the filter sits
The reproduction is a trajectory-level filter, not a command-level one: it reshapes the planner’s whole reference horizon before the controller ever sees it, so the safety guarantee is carried by the reference the car tracks.
flowchart LR
T[("Track / Road<br/>tracklib")] --> PL["planner<br/>→ reference horizon"]
PL --> OT["overtake state<br/>FOLLOW · OVERTAKE · MERGE"]
OT --> SF["DR-CVaR filter<br/>per-node separating halfspace<br/>(CVaR ≤ δ over a Wasserstein ball)"]
PRED[("obstacle prediction<br/>+ ambiguity set")] -.-> SF
SF --> C["controller<br/>tracks the safe reference"]
C --> CMD["command → drive"] --> R[("record<br/>MCAP")]
classDef lib fill:#edece7,stroke:#8a857c,color:#2a2824;
classDef algo fill:#ffe6d5,stroke:#ff5400,stroke-width:2px,color:#7a2e00;
class T,PRED,R lib;
class SF algo;
A consequence worth stating: because the guarantee lives in the reference, a physical car’s tracking error has to fit inside whatever clearance the reference actually carries — and that budget is far smaller than it first looks. It is tempting to score a 20–30 cm tracking error against the filter’s 0.60 m padding and conclude there is plenty of room. But that padding is not slack — it is the collision distance itself, the two bodies’ radii summed, and it is fully consumed by geometry the instant the cars are abeam. The tracker’s real budget is only what the reference clears on top of that padding — here about +21 cm — so measuring tracking error against 60 cm overstated the margin by roughly 3×. That tracking-margin axis is one of the honest costs the reproduction surfaces, alongside solve time (every halfspace is its own convex program).
The third cost is clearance over-provisioning. Under the paper’s own predictor settings, the DR-CVaR
safe reference bows about 1.6 m off centerline on the overtake — more than twice a lab lane’s
half-width. On a narrow track that extra caution pushes the safe reference off the lane outright, so real
geometry forces either a (σ, ε, r) retune or a braking fallback: the guarantee is not free on a narrow
track. The honest nuance is that the module actually ran in open motion-capture space, where the lane is
a projection on the floor, not a barrier — there a 1.6 m bow is free room, not a wall strike. But a finite
lab still has to physically contain that bow, so it is a real space budget, looser than a lane bound
and differently shaped, and it is what motivates a dedicated overtake-lane formulation as the next step.
The decisions, and what they cost
Reproducing a paper is a sequence of choices about where to be faithful and where the platform forces you to be honest instead. These are the four that decided what the numbers mean.
Trade-offs — what was chosen, and what it costs
What counts as a collision?
A disc cannot tell abeam from nose-to-tail, which is the entire distinction an overtake is made of, and it over-reports lateral clearance by roughly 2×. Being faithful to the paper here would have meant reporting a safety margin that the geometry does not support. The cost is real and I’d rather pay it than quote a comparable-looking number that measures the wrong thing.
What does an unstamped state mean?
Time is carried as a first-class field on every state and trajectory point. The alternative — a
silent 0.0 — is what let a timebase skew produce a confident wrong answer seven separate times.
A filter that refuses to answer is strictly more useful than one that answers optimistically.
Score against the estimate, or the truth?
Scoring a safety filter on its own belief measures self-consistency, not safety. Keeping the evaluation on ground truth is what makes a passing number mean something, and it deliberately gives up the ability to run the same metric live on the car.
Is hardware required for the reproduction to count?
The criteria were ratified before the results, and hardware deliberately isn’t one of them: the questions that decide whether this reproduction is honest — is the ambiguity sized from measured residuals, is the predicate right, is the timebase sound — are all answerable in simulation, and answering them there is higher leverage than rushing onto the rig. What that buys is a real limit, stated as one.
Does the guarantee survive the move to a tracked car?
The filter certifies a reference. A real car does not ride the reference — a controller tracks it, imperfectly. So the honest question, and the one the paper’s holonomic plant never has to ask, is whether the certificate survives the tracked closed loop. I tested it directly: certify a safe reference, hand it to a tracker, and score the tracked trajectory on the honest footprint predicate.
The measured answer, on this scenario, is no — not for the reference alone. Handed the same certified reference, an MPC tracker and a PID tracker both drive the car into the obstacle’s footprint, by −57.8 cm and −51.2 cm respectively. The reason is structural and, once seen, obvious: a DR-CVaR certificate is time-parameterized — it guarantees a separating halfspace at each instant of the plan. Tracking a path is not tracking a schedule. A controller that follows the right geometry a beat late arrives at the right place at the wrong time, and the guarantee was a statement about time.

Why the certificate does not transfer, split across the two axes that decide it. Left: cross-track error against the reference’s real slack — the dotted line at just 10.5 cm, not the 60 cm padding (which is collision distance, not budget). PID overruns it; MPC fits under it and breaches anyway. Right: the axis nobody was watching — schedule lag. Neither tracker holds the reference’s timing (MPC falls 4.15 m behind, PID 7.25 m), and the guarantee is a statement about where the obstacle is at time k·dt. Geometry inside budget is not enough; the schedule is the axis the transfer fails on.
What rescues it is not a better reference but a better loop. Re-running the filter every control step — re-establishing the certificate against the car’s actual current state instead of trusting a plan computed once — clears the obstacle by +15 cm, and is the only arm that does. The telling ordering is that a stale one-shot plan tracked open-loop is worse than running no filter at all: one-shot < none < receding. Reacting live beats executing a safe-but-stale plan.
That is the finding I’d keep above all the others: the guarantee is not a property of the safe reference — it is a property of the closed loop that keeps re-establishing it. A safety filter you solve once and then track open-loop is a safety filter you do not actually have.
The verification story — one bug, seven disguises
The part I’m most willing to show is not a result but a process. Reproducing a safety filter means the metric that says “safe” has to be trustworthy, and the failure mode of a safety reproduction is a number that looks right for the wrong reason. Over the arc I found and fixed seven separate instances of a single bug class: an ego state and an obstacle prediction being compared across a timebase skew — paired one control step off, cross-checked against the wrong reference line, or scored on a belief instead of the true pose. Each instance passed its tests. Each reported a plausible clearance. The seventh had reached the multi-agent capstone before it was caught.
flowchart TB
subgraph BUG["THE SKEW · 7 disguises, every test green"]
direction LR
be["ego pose<br/>@ t+dt"] --> bscore{"score<br/>collision"}
bo["obstacle pred<br/>@ t (by list index)"] --> bscore
bscore --> bout["plausible clearance —<br/>silently one step optimistic"]
end
subgraph FIX["THE FIX · make the skew unrepresentable"]
direction LR
fe["ego pose<br/>+ carried time"] --> gate{"same-instant<br/>gate"}
fo["obstacle pred<br/>resolved by time"] --> gate
gate -->|"instants agree"| ok["score on the TRUE pose"]
gate -->|"mismatch / unstamped"| no["REFUSE to plan<br/>(nan, never a silent 0.0)"]
end
BUG -. "structural fix,<br/>not 7 patches" .-> FIX
classDef ctx fill:#edece7,stroke:#8a857c,color:#2a2824;
classDef bad fill:#fbe3e0,stroke:#c0392b,color:#7a2018;
classDef good fill:#ffe6d5,stroke:#ff5400,stroke-width:2px,color:#7a2e00;
class be,bo,fe,fo ctx;
class bscore,bout bad;
class gate,ok,no good;
The fixes hardened the whole thing: time is now carried as a first-class field on every state and
trajectory point (an unstamped one is nan, never silently 0.0), a same-instant gate refuses to
plan on skewed inputs rather than producing a confident wrong answer, and every metric is scored from
the true pose, never the estimate. The other methodological correction was the collision predicate
itself: the paper pads obstacles into an isotropic 2·radius = 0.60 m disc, which over-reports
lateral clearance by ~2× — two 0.30 m-wide cars pass abeam at 0.30 m centre-to-centre, not 0.60 m. The
honest predicate is the oriented footprint gap (a separating-axis test over the two rectangles); a
disc cannot tell “abeam” from “nose-to-tail,” which is the entire distinction an overtake is made of.
One earlier confound belongs to the same family, even though it isn’t a timebase skew: the first tracking harness re-anchored a fresh safe reference every step, so the “tracking error” it reported was really the ego’s own excursion off a path that kept sliding out from under it — not tracking fidelity at all. Like all seven, it was caught by asking what a number actually compared, not by a failing test; the suite was green the whole time. That is the throughline: none of these were found by a red test, because a metric that pairs the wrong things still computes — the only defense is auditing the instrument itself.
Findings — the instances, pinned
A plausible clearance, reported by a green suite, at the multi-agent capstonefixed
Six instances had already been fixed one at a time. The seventh is the one that proved patching instances was the wrong response — the class had to be made unrepresentable, not enumerated. All nine reported numbers came out bit-identical after the change, which is the only reason I trust the fix didn’t quietly move the results.
Lateral clearance that looked comfortable in every overtakefixed
Not a bug in the code — a bug in the instrument. The implementation faithfully computed the quantity it was asked for; the quantity was the wrong one. Re-scoring on the footprint moved the raw maneuver margin to +0.148 m and the guarded one to +0.339 m, which is also where the headline result finally became legible: the guard roughly doubles clearance.
A tracking-error curve that never settled, on a controller that visibly tracked wellfixed
Structurally the same family as the seven — a number comparing two things that were never meant to be compared — even though it isn’t a timebase skew. It reinforces the throughline: not one of these was found by a red test, because a metric that pairs the wrong things still computes cleanly. The only defence is auditing the instrument itself.
The same-instant discipline holds at the boundary, but nothing checks it downstreamopen
Recorded rather than quietly carried. The invariant is currently maintained by construction and by one gate, which is exactly the shape the original bug had before it had seven disguises.
Limitations, kept first-class
The paper’s holonomic model hides real-platform limits the reproduction keeps out in the open rather than tuning away:
- DR-CVaR alone cannot make an in-lane overtake from a head-on reference under a speed cap — the separating normal for a near-collinear obstacle is almost purely longitudinal, so dead head-on gives no lateral trigger and the filter brakes into a collision. The answer is a committed maneuver the filter then guards.
- The conservative arms go infeasible without lateral room — near head-on, DR-CVaR’s larger separation is unachievable under a real actuator box.
- The ambiguity growth is sized for one operating speed — measured on the platform’s own predictor, the growth is speed-dependent (~v²/R); it’s honest at the nominal 1.0 m/s obstacle and deliberately not over-sized for a faster opponent (over-padding is its own trap). A faster-opponent scenario needs a re-sized growth.
What’s next
The sim-answerable maturity is done: the predictor’s ambiguity is sized from its own measured residuals, the ego/mocap latency sensitivity is characterized (on the near-ground-truth OptiTrack rig, latency is a portability axis, not an operational budget), and the solver’s first-call cost is pre-warmed out. The remaining rungs are hardware-gated — wiring the filter into the real-time loop on the car and re-measuring the timing budget on the target ECU. That’s the virtual_track VT5 step: DR-CVaR becomes CONLab’s reusable safety-filter layer — any controller made robust to prediction uncertainty, with the observability to prove it.
Updates
Reverse-chronological log below the evergreen case study.
- 2026-07-19 — R9 (ego/mocap latency) closed in sim as a portability characterization; the honest-demo capstone committed. Case study added to the portfolio as a WIP entry.
- 2026-07-18 — Honest-demo arc: car-like model-mismatch fix, oriented-footprint predicate, explicit overtake state, live-run halfspace render, and the CVaR-vs-DR-CVaR overlay (constant 0.25 m margin). Both single-lane and two-lane overtakes clear cleanly.
- 2026-07-16 — The seventh and final timebase-bug instance found — in the filter’s own pairing math; time promoted to a carried, first-class field across the state/trajectory contract.
Full spec
- Stack
- Python 3.11 · cvxpy · CLARABEL · casadi · numpy · Foxglove / MCAP · f1tenth_gym · pixi
- Hardware
- COMO · OptiTrack motion capture · (hardware bridge — next rung)
- Collaborators
- Sleiman Safaoui & Tyler Summers (original paper, CONLab / UT Dallas)