Contents

robotics · active

DR-CVaR — reproducing a distributionally-robust safety filter, honestly

A clean-room reproduction of my lab's ICRA'24 distributionally-robust CVaR safety filter, moved off the paper's idealized plant onto a car-like platform — and the verification story of what actually survives the move.

DR-CVaR is a robustness margin on a committed maneuver, not the thing that makes the pass possible — and the real guarantee lives in the closed loop that keeps re-establishing it, not in the safe reference you compute once.

Work in progressLive, but still being revised — the writing, diagrams, and data figures are all actively being critiqued and improved.

Role
Sole author of the clean-room reproduction — I stood up the original DR-CVaR filter at CONLab and rebuilt it ROS-free on the conlab v2 stack, then ran the honest-demo verification arc.
When
Original at CONLab · v2 reproduction 2026-06 – present
  • arXiv:2309.08821Paper reproduced
  • CVaR 29/30 → DR-CVaR 12/30Distribution shift, over-bound rate
  • 7Verification-bug instances found + fixed
  • oriented footprint, not discHonest collision predicate
  • 128Tests green
  • hardware (VT5)Next rung

The paper, and why reproduce it

A safety filter sits between a controller and the plant: the controller proposes a motion, the filter minimally reshapes it to stay safe, and the reshaped command is what drives. The appeal is separation — you can develop an aggressive planner and bolt a safety guarantee on afterward, rather than baking caution into every controller.

The filter I reproduced is my lab’s own: Distributionally Robust CVaR-Based Safety Filtering (Safaoui & Summers, ICRA 2024, arXiv:2309.08821). Its idea is to be robust not just to an obstacle’s predicted uncertainty, but to the prediction being the wrong distribution. Instead of trusting a single Gaussian forecast of where the obstacle will be, it solves for a separating halfspace that keeps the Conditional Value-at-Risk of a collision under a bound for every distribution within a Wasserstein ball around the forecast. You pay for the distribution you didn’t model.

I built the original version of this filter at CONLab during my bachelor’s. Rebuilding it clean-room — ROS-free, on the conlab v2 stack — was a chance to do something more useful than re-derive the math: to move it off the paper’s idealized plant and find out what the guarantee is actually worth on a car.

The honest headline — a margin guard, not a pass-maker

The paper’s filter is validated on a 10 g holonomic double integrator — a point mass that can accelerate in any direction, hard. A COMO RC car is a car-like bicycle running ~1.25 m/s with a ~0.5 g acceleration box and a steering constraint. That gap is not cosmetic. Under a real actuator box, a head-on centerline reference has no lateral escape and simply collides — the filter, on its own, cannot invent an in-lane overtake from a head-on reference. So the honest scenario is a guarded maneuver: the ego commits an explicit FOLLOW → OVERTAKE → MERGE state, and DR-CVaR guards that committed bow against prediction uncertainty.

An oval track in the motion-capture frame. The obstacle (red dashed) holds the centerline; the ego (green) bows outward around it and clears by 0.29 m at closest approach. The DR-CVaR separating halfspace is drawn in orange with its safe side shaded, the plain-CVaR separator in dashed purple just inside it, labeled 'DR margin 0.25 m', and the obstacle's 1σ ambiguity set as a small red ellipse.

The whole filter in one frame, drawn on a real run at the decisive instant. Orange is the DR-CVaR separating halfspace and its safe side; dashed purple is the plain-CVaR separator it would have used with no distributional robustness. The gap between them — a constant 0.25 m here — is exactly what the Wasserstein ball buys. The pass clears by 0.29 m on the oriented-footprint gap between the two 0.60 × 0.30 m bodies.

This reframes the result rather than weakening it. On the honest footprint predicate (below) the raw committed maneuver already clears the obstacle by +0.148 m; the DR-CVaR guard roughly doubles the clearance to +0.339 m. So DR-CVaR’s contribution on a realistic platform is a robustness margin on a maneuver the ego commits — not the thing that makes the pass possible. Saying that plainly, with the number, is the contribution: the paper’s headline was measured on a plant no RC car can drive, and this reproduction shows exactly what survives on one that can.

What distributional robustness actually buys

The clearest positive result is a distribution-shift stress test. Each separator is designed against a Gaussian obstacle cloud, then the obstacle is realized under a heavier-tailed Laplace at the same centre and scale — a deliberate mismatch between the modelled and the true distribution. The plain-CVaR separator, which trusts its Gaussian, breaches its own risk bound almost every time. The DR-CVaR separator, hedged over a Wasserstein ball, breaches it far less.

A strip plot with two columns. Left, CVaR (eps=0): 29 of 30 realized-risk points sit above the delta bound of 0.1. Right, DR-CVaR (eps=0.1): only 12 of 30 sit above the bound, most clustered well below it.

Robustness to the wrong distribution, across 30 seeds. Designed on a Gaussian, realized under a Laplace tail: plain CVaR exceeds its risk bound 29/30 times, DR-CVaR only 12/30. This is the property the whole method exists to buy, and it reproduces cleanly.

Underneath it is a cleaner invariant that also reproduces: the conservatism ordering Mean ≤ CVaR ≤ DR-CVaR in the halfspace offset, monotone in the Wasserstein radius. DR-CVaR is always at least as cautious as CVaR, which is at least as cautious as the mean separator.

Where the filter sits

The reproduction is a trajectory-level filter, not a command-level one: it reshapes the planner’s whole reference horizon before the controller ever sees it, so the safety guarantee is carried by the reference the car tracks.

flowchart LR
  T[("Track / Road<br/>tracklib")] --> PL["planner<br/>→ reference horizon"]
  PL --> OT["overtake state<br/>FOLLOW · OVERTAKE · MERGE"]
  OT --> SF["DR-CVaR filter<br/>per-node separating halfspace<br/>(CVaR ≤ δ over a Wasserstein ball)"]
  PRED[("obstacle prediction<br/>+ ambiguity set")] -.-> SF
  SF --> C["controller<br/>tracks the safe reference"]
  C --> CMD["command → drive"] --> R[("record<br/>MCAP")]
  classDef lib fill:#edece7,stroke:#8a857c,color:#2a2824;
  classDef algo fill:#ffe6d5,stroke:#ff5400,stroke-width:2px,color:#7a2e00;
  class T,PRED,R lib;
  class SF algo;

A consequence worth stating: because the guarantee lives in the reference, a physical car’s tracking error has to fit inside whatever clearance the reference actually carries — and that budget is far smaller than it first looks. It is tempting to score a 20–30 cm tracking error against the filter’s 0.60 m padding and conclude there is plenty of room. But that padding is not slack — it is the collision distance itself, the two bodies’ radii summed, and it is fully consumed by geometry the instant the cars are abeam. The tracker’s real budget is only what the reference clears on top of that padding — here about +21 cm — so measuring tracking error against 60 cm overstated the margin by roughly 3×. That tracking-margin axis is one of the honest costs the reproduction surfaces, alongside solve time (every halfspace is its own convex program).

The third cost is clearance over-provisioning. Under the paper’s own predictor settings, the DR-CVaR safe reference bows about 1.6 m off centerline on the overtake — more than twice a lab lane’s half-width. On a narrow track that extra caution pushes the safe reference off the lane outright, so real geometry forces either a (σ, ε, r) retune or a braking fallback: the guarantee is not free on a narrow track. The honest nuance is that the module actually ran in open motion-capture space, where the lane is a projection on the floor, not a barrier — there a 1.6 m bow is free room, not a wall strike. But a finite lab still has to physically contain that bow, so it is a real space budget, looser than a lane bound and differently shaped, and it is what motivates a dedicated overtake-lane formulation as the next step.

The decisions, and what they cost

Reproducing a paper is a sequence of choices about where to be faithful and where the platform forces you to be honest instead. These are the four that decided what the numbers mean.

Trade-offs — what was chosen, and what it costs

What counts as a collision?

chose
Oriented-footprint gap — a separating-axis test over two rectangles
over
the paper’s isotropic 2·radius = 0.60 m disc
costs
a departure from the paper's own predicate, so my clearance numbers are not directly comparable to its published ones

A disc cannot tell abeam from nose-to-tail, which is the entire distinction an overtake is made of, and it over-reports lateral clearance by roughly 2×. Being faithful to the paper here would have meant reporting a safety margin that the geometry does not support. The cost is real and I’d rather pay it than quote a comparable-looking number that measures the wrong thing.

What does an unstamped state mean?

chose
nan, and a same-instant gate that refuses to plan
over
defaulting to 0.0 · back-filling the missing timestamp at the boundary
costs
a class of runs now fails loudly that used to complete — and the refusal is at the seam, so a caller has to handle it

Time is carried as a first-class field on every state and trajectory point. The alternative — a silent 0.0 — is what let a timebase skew produce a confident wrong answer seven separate times. A filter that refuses to answer is strictly more useful than one that answers optimistically.

Score against the estimate, or the truth?

chose
Every metric scored from the true pose
over
scoring on the filter’s own belief · scoring on the published estimate
costs
the metric can no longer be computed online on hardware, where no true pose exists — it is an offline evaluation instrument only

Scoring a safety filter on its own belief measures self-consistency, not safety. Keeping the evaluation on ground truth is what makes a passing number mean something, and it deliberately gives up the ability to run the same metric live on the car.

Is hardware required for the reproduction to count?

chose
Sim-answerable maturity as the success criterion; hardware as a separate rung
over
gating the whole reproduction on the physical car
costs
no claim here is validated against real actuator, latency, or tyre behaviour — the sim-to-real gap is unmeasured, not closed

The criteria were ratified before the results, and hardware deliberately isn’t one of them: the questions that decide whether this reproduction is honest — is the ambiguity sized from measured residuals, is the predicate right, is the timebase sound — are all answerable in simulation, and answering them there is higher leverage than rushing onto the rig. What that buys is a real limit, stated as one.

Does the guarantee survive the move to a tracked car?

The filter certifies a reference. A real car does not ride the reference — a controller tracks it, imperfectly. So the honest question, and the one the paper’s holonomic plant never has to ask, is whether the certificate survives the tracked closed loop. I tested it directly: certify a safe reference, hand it to a tracker, and score the tracked trajectory on the honest footprint predicate.

The measured answer, on this scenario, is no — not for the reference alone. Handed the same certified reference, an MPC tracker and a PID tracker both drive the car into the obstacle’s footprint, by −57.8 cm and −51.2 cm respectively. The reason is structural and, once seen, obvious: a DR-CVaR certificate is time-parameterized — it guarantees a separating halfspace at each instant of the plan. Tracking a path is not tracking a schedule. A controller that follows the right geometry a beat late arrives at the right place at the wrong time, and the guarantee was a statement about time.

Two panels sharing a time axis. Left: cross-track error for MPC (blue, peak 0.14 m) and PID (orange dashed, peak 0.34 m) against a dark-red dotted line at 0.105 m marked 'actual slack (r_pad is NOT slack)'; PID sails over it, MPC mostly stays under it. Right: schedule lag along the path, MPC peaking at 4.15 m and PID at 7.25 m — the uncounted axis. Both trackers breach the collision distance despite MPC's cross-track error fitting under the slack.

Why the certificate does not transfer, split across the two axes that decide it. Left: cross-track error against the reference’s real slack — the dotted line at just 10.5 cm, not the 60 cm padding (which is collision distance, not budget). PID overruns it; MPC fits under it and breaches anyway. Right: the axis nobody was watching — schedule lag. Neither tracker holds the reference’s timing (MPC falls 4.15 m behind, PID 7.25 m), and the guarantee is a statement about where the obstacle is at time k·dt. Geometry inside budget is not enough; the schedule is the axis the transfer fails on.

What rescues it is not a better reference but a better loop. Re-running the filter every control step — re-establishing the certificate against the car’s actual current state instead of trusting a plan computed once — clears the obstacle by +15 cm, and is the only arm that does. The telling ordering is that a stale one-shot plan tracked open-loop is worse than running no filter at all: one-shot < none < receding. Reacting live beats executing a safe-but-stale plan.

That is the finding I’d keep above all the others: the guarantee is not a property of the safe reference — it is a property of the closed loop that keeps re-establishing it. A safety filter you solve once and then track open-loop is a safety filter you do not actually have.

The verification story — one bug, seven disguises

The part I’m most willing to show is not a result but a process. Reproducing a safety filter means the metric that says “safe” has to be trustworthy, and the failure mode of a safety reproduction is a number that looks right for the wrong reason. Over the arc I found and fixed seven separate instances of a single bug class: an ego state and an obstacle prediction being compared across a timebase skew — paired one control step off, cross-checked against the wrong reference line, or scored on a belief instead of the true pose. Each instance passed its tests. Each reported a plausible clearance. The seventh had reached the multi-agent capstone before it was caught.

flowchart TB
  subgraph BUG["THE SKEW · 7 disguises, every test green"]
    direction LR
    be["ego pose<br/>@ t+dt"] --> bscore{"score<br/>collision"}
    bo["obstacle pred<br/>@ t &nbsp;(by list index)"] --> bscore
    bscore --> bout["plausible clearance —<br/>silently one step optimistic"]
  end
  subgraph FIX["THE FIX · make the skew unrepresentable"]
    direction LR
    fe["ego pose<br/>+ carried time"] --> gate{"same-instant<br/>gate"}
    fo["obstacle pred<br/>resolved by time"] --> gate
    gate -->|"instants agree"| ok["score on the TRUE pose"]
    gate -->|"mismatch / unstamped"| no["REFUSE to plan<br/>(nan, never a silent 0.0)"]
  end
  BUG -. "structural fix,<br/>not 7 patches" .-> FIX
  classDef ctx fill:#edece7,stroke:#8a857c,color:#2a2824;
  classDef bad fill:#fbe3e0,stroke:#c0392b,color:#7a2018;
  classDef good fill:#ffe6d5,stroke:#ff5400,stroke-width:2px,color:#7a2e00;
  class be,bo,fe,fo ctx;
  class bscore,bout bad;
  class gate,ok,no good;

The fixes hardened the whole thing: time is now carried as a first-class field on every state and trajectory point (an unstamped one is nan, never silently 0.0), a same-instant gate refuses to plan on skewed inputs rather than producing a confident wrong answer, and every metric is scored from the true pose, never the estimate. The other methodological correction was the collision predicate itself: the paper pads obstacles into an isotropic 2·radius = 0.60 m disc, which over-reports lateral clearance by ~2× — two 0.30 m-wide cars pass abeam at 0.30 m centre-to-centre, not 0.60 m. The honest predicate is the oriented footprint gap (a separating-axis test over the two rectangles); a disc cannot tell “abeam” from “nose-to-tail,” which is the entire distinction an overtake is made of.

One earlier confound belongs to the same family, even though it isn’t a timebase skew: the first tracking harness re-anchored a fresh safe reference every step, so the “tracking error” it reported was really the ego’s own excursion off a path that kept sliding out from under it — not tracking fidelity at all. Like all seven, it was caught by asking what a number actually compared, not by a failing test; the suite was green the whole time. That is the throughline: none of these were found by a red test, because a metric that pairs the wrong things still computes — the only defense is auditing the instrument itself.

Findings — the instances, pinned

A plausible clearance, reported by a green suite, at the multi-agent capstonefixed

cause
The seventh instance of the skew was in the filter's own pairing math — an ego state compared against an obstacle prediction resolved by list index rather than by time
fix
Time promoted to a carried field across the whole state/trajectory contract; pairing resolved by instant, never by index
ref
conlab · 9ed908c

Six instances had already been fixed one at a time. The seventh is the one that proved patching instances was the wrong response — the class had to be made unrepresentable, not enumerated. All nine reported numbers came out bit-identical after the change, which is the only reason I trust the fix didn’t quietly move the results.

Lateral clearance that looked comfortable in every overtakefixed

cause
The paper's isotropic 0.60 m disc pad over-reports abeam clearance by ~2× — two 0.30 m-wide cars pass at 0.30 m centre-to-centre, not 0.60 m
fix
Oriented-footprint separating-axis predicate, with grading, replacing the disc
ref
conlab · footprint predicate

Not a bug in the code — a bug in the instrument. The implementation faithfully computed the quantity it was asked for; the quantity was the wrong one. Re-scoring on the footprint moved the raw maneuver margin to +0.148 m and the guarded one to +0.339 m, which is also where the headline result finally became legible: the guard roughly doubles clearance.

A tracking-error curve that never settled, on a controller that visibly tracked wellfixed

cause
The first tracking harness re-anchored a fresh safe reference every step, so the 'error' was the ego's excursion off a path that kept sliding out from under it
fix
Reference anchored once per evaluation window; the metric measures tracking fidelity again

Structurally the same family as the seven — a number comparing two things that were never meant to be compared — even though it isn’t a timebase skew. It reinforces the throughline: not one of these was found by a red test, because a metric that pairs the wrong things still computes cleanly. The only defence is auditing the instrument itself.

The same-instant discipline holds at the boundary, but nothing checks it downstreamopen

cause
The gate lives at the egress seam; no consumer independently enforces that the states it received actually share an instant
fix
Not yet built — a consumer-side assertion is the obvious next rung

Recorded rather than quietly carried. The invariant is currently maintained by construction and by one gate, which is exactly the shape the original bug had before it had seven disguises.

Limitations, kept first-class

The paper’s holonomic model hides real-platform limits the reproduction keeps out in the open rather than tuning away:

  1. DR-CVaR alone cannot make an in-lane overtake from a head-on reference under a speed cap — the separating normal for a near-collinear obstacle is almost purely longitudinal, so dead head-on gives no lateral trigger and the filter brakes into a collision. The answer is a committed maneuver the filter then guards.
  2. The conservative arms go infeasible without lateral room — near head-on, DR-CVaR’s larger separation is unachievable under a real actuator box.
  3. The ambiguity growth is sized for one operating speed — measured on the platform’s own predictor, the growth is speed-dependent (~v²/R); it’s honest at the nominal 1.0 m/s obstacle and deliberately not over-sized for a faster opponent (over-padding is its own trap). A faster-opponent scenario needs a re-sized growth.

What’s next

The sim-answerable maturity is done: the predictor’s ambiguity is sized from its own measured residuals, the ego/mocap latency sensitivity is characterized (on the near-ground-truth OptiTrack rig, latency is a portability axis, not an operational budget), and the solver’s first-call cost is pre-warmed out. The remaining rungs are hardware-gated — wiring the filter into the real-time loop on the car and re-measuring the timing budget on the target ECU. That’s the virtual_track VT5 step: DR-CVaR becomes CONLab’s reusable safety-filter layer — any controller made robust to prediction uncertainty, with the observability to prove it.

Updates

Reverse-chronological log below the evergreen case study.

  • 2026-07-19 — R9 (ego/mocap latency) closed in sim as a portability characterization; the honest-demo capstone committed. Case study added to the portfolio as a WIP entry.
  • 2026-07-18 — Honest-demo arc: car-like model-mismatch fix, oriented-footprint predicate, explicit overtake state, live-run halfspace render, and the CVaR-vs-DR-CVaR overlay (constant 0.25 m margin). Both single-lane and two-lane overtakes clear cleanly.
  • 2026-07-16 — The seventh and final timebase-bug instance found — in the filter’s own pairing math; time promoted to a carried, first-class field across the state/trajectory contract.
Full spec
Stack
Python 3.11 &middot; cvxpy &middot; CLARABEL &middot; casadi &middot; numpy &middot; Foxglove / MCAP &middot; f1tenth_gym &middot; pixi
Hardware
COMO &middot; OptiTrack motion capture &middot; (hardware bridge — next rung)
Collaborators
Sleiman Safaoui & Tyler Summers (original paper, CONLab / UT Dallas)