Vascora is a reproducible reinforcement-learning study measuring how the weight of a filtered vessel-wall contact-force penalty trades simulated guidewire contact force against navigation success in CathSim — swept across five penalty weights and five random seeds, not a single run at a single setting.
In simple terms: can an AI guidewire learn to reach a target branch more gently inside a physics simulation?
Does adding a filtered guidewire–vessel-wall contact-force penalty to CathSim's navigation reward reduce cumulative and peak simulated contact force during Type-I aortic arch navigation, without substantially reducing PPO task success?
The site follows the structure of a research paper: problem → related work → protocol → mathematics → experiment plan → results → failure modes → limitations.
Why a reward that only values arriving produces collision-heavy navigation.
Read →CathSim, MuJoCo, PPO, reward shaping and contact force in plain language.
Read →What the prior literature reports, what it leaves open, and the one sentence this study is allowed to claim.
Read →The question, the hypothesis, and the decisions fixed before training.
Read →The MDP, the reward equation, and the planned hyperparameters.
Read →The learning loop and the sim-to-real boundary.
Read →Interactive: reward code, equation and trajectory, side by side.
Open →The λ sweep, the seed protocol, the statistics and the compute budget.
Read →The safety dashboard and trade-off curve — specified before any data exists.
Read →Six named failure modes, defined in advance so none can be hidden later.
Read →What this study explicitly does not claim.
Read →Environment pinning, run artefacts, commit discipline, dated log and citation.
Read →The researcher and planned future directions.
Read →Endovascular navigation steers a guidewire through curved, branching vessels toward a target. An agent rewarded only for arriving has no reason to avoid pressing on the vessel wall.
In real procedures, excessive guidewire–wall contact is associated with tissue trauma. Any autonomous system must therefore be judged on how it navigates, not only whether it arrives.
Standard navigation rewards optimise progress, a goal bonus and a step cost. Contact is unpriced, so scraping along the wall can be a perfectly good strategy for the agent.
Two policies with identical success rates can differ enormously in force. Success rate alone cannot separate them — contact must be measured directly.
Reinforcement learning is relentless about optimising exactly what you measure. If the only thing that earns reward is reaching the target quickly, an agent will discover that pressing against the vessel wall is a perfectly good shortcut — nothing in its objective says otherwise. The behaviour is “successful”, but rough. To get gentleness, gentleness has to be written into the reward.
Vascora does not claim clinical safety. It tests whether a force-related safety proxy improves behaviour inside one controlled simulation environment.
A short glossary of every term this project uses — no prior background assumed.
An open-source research simulator for catheter and guidewire navigation inside blood vessels. It lets an agent practise navigation without any real patient, and reports when and where the guidewire touches the vessel.
The physics engine underneath CathSim. It computes how the guidewire bends, advances and rotates, when it collides with the wall, and how much force each contact produces.
A thin, flexible wire steered through vessels toward a target, used clinically to guide other instruments. Here it is entirely virtual.
The curved main artery with branches toward the head and arms. The agent must steer to one branch — the brachiocephalic artery (BCA) — passing the other two origins on the way.
Proximal Policy Optimization: a stable, widely used policy-gradient method for continuous control. Chosen because it is well understood, so the experiment isolates the reward change rather than the algorithm.
Adjusting the score an agent maximises so it learns better behaviour. Here: reach the target, and avoid pressing hard on the wall.
The force produced when the guidewire touches the wall. Reported two ways: cumulative (total across an episode, a measure of overall gentleness) and peak (the single hardest moment).
A penalty that ignores contact below a threshold τ and grows steeply above it — so unavoidable light touches are free, but sustained pressure is expensive.
Two agents identical in every respect except the reward. Only the force-aware agent loses reward for strong vessel-wall contact.
An episode is one complete navigation attempt; a step is one small action; a seed fixes the randomness so a run can be repeated exactly.
Removing or varying one component to show it is responsible for an effect. Here: training with λ = 0 and with several penalty strengths.
Testing a trained policy on anatomies it never saw during training — evidence that it learned navigation rather than memorising one geometry.
Penalty-based reward shaping is standard, and learning-based catheter navigation already exists. This section says what the prior work reports, what it does not, and therefore what this study can honestly claim to add.
Across the endovascular navigation literature I could verify, the contact-force penalty weight is never treated as an isolated manipulated variable — where a force term enters the reward at all it is one hand-chosen constant, ablated on or off rather than swept — so the contact-force-versus-navigation-success trade-off curve, with algorithm, environment, observation space, network and training budget held fixed and variance reported across random seeds, has not been plotted in any study I could find.
Penalty-based reward shaping is textbook, and force-aware reinforcement learning for catheter navigation already exists — I claim neither. Robertshaw et al. fold guidewire tip force directly into a soft actor–critic reward and report 0.24 N mean force at 96% success against a 1.5 N rupture threshold [7]; Song et al. put an F/T-sensor force term in a physical robot’s reward [15]; the CathSim line measures mean and peak catheter–aorta force [1,4] and defines a 2 N force-thresholded safety metric [2]; and mean, maximum and time-integrated contact force were established as endovascular safety descriptors long before any of that [17]. Multi-seed training is prior art here too, since CathSim’s expert-trajectory work already used five random seeds [2]. The simulator, the contact model, the metrics and the idea of penalising contact are all someone else’s.
What I could not find anywhere in this literature is the force-penalty weight treated as an isolated manipulated variable. In the only two studies I could verify where a force term enters the reward at all, that weight is a single hand-chosen constant: [7] reports one −0.01 coefficient above a 0.85 N threshold and varies the term’s presence rather than its magnitude, and [15] fixes its weights outright, so each yields one operating point rather than a curve. Elsewhere force is measured but never optimised [1,2,4,11,14], replaced by a geometric wall-distance proxy [13], or absent entirely [6,8,10,12]; a mechanical-thrombectomy navigation study names a threshold-based contact-force penalty explicitly as future work it did not do [9], and a recent open simulation framework lists device–wall contact force measurement as a capability it does not yet have [5]. Variance across training runs is also largely unreported: in [1,4,5,7,14] the ± figures are spread over evaluation episodes of a single trained agent, and a 2025 benchmark in this exact task domain states outright that it lacked systematic random-seed exploration [6]. Seeds are therefore the floor, not the finding. My contribution is calibration rather than invention: sweep that one coefficient with algorithm, environment, observation space, network and training budget frozen, repeat every setting across seeds, and plot the contact-force-versus-success curve that none of the studies I could verify plots. If that curve turns out flat, that is the result.
The gap above is a claim about a table, not a feeling. Each row was read off the paper itself; “unclear” means the paper does not say, and is left as unclear rather than resolved in this project’s favour. Every reference was checked against a fetched record — not one was accepted on memory.
| Prior work | Reports a contact-force metric | Seeds reported | Sweeps a penalty coefficient | Plots a force–success trade-off |
|---|---|---|---|---|
| [1] CathSim: An Open-Source Simulator for Endovascular Intervention 2024 | yes | not stated | no | no |
| [2] Autonomous Catheterization with Open-source Simulator and Expert Trajectory 2024 | yes | 5 | no | no |
| [3] Towards Autonomous Navigation in Endovascular Interventions 2025 | yes | 5 † | unclear | unclear |
| [4] CathSim — open-source simulator for endovascular intervention (software repository, airvlab/cathsim) 2023 | yes | not stated | no | no |
| [5] Learning-based autonomous navigation, benchmark environments and simulation framework for endovascular interventions 2025 | no | not stated | no | no |
| [6] Benchmarking reinforcement learning algorithms for autonomous mechanical thrombectomy 2025 | no | 1 † | no | no |
| [7] Reinforcement learning for safe autonomous two-device navigation of cerebral vessels in mechanical thrombectomy 2025 | yes | not stated | no | no |
| [8] A zero-shot reinforcement learning strategy for autonomous guidewire navigation 2024 | no | not stated | no | no |
| [9] Autonomous navigation of catheters and guidewires in mechanical thrombectomy using inverse reinforcement learning 2024 | no | not stated | no | no |
| [10] Learning-based autonomous vascular guidewire navigation without human demonstration in the venous system of a porcine liver 2022 | no | not stated | no | no |
| [11] Collaborative Robot-Assisted Endovascular Catheterization with Generative Adversarial Imitation Learning 2020 | yes | not stated | no | no |
| [12] Deep Reinforcement Learning for Guidewire Navigation in Coronary Artery Phantom 2021 | no | not stated | no | no |
| [13] Autonomous Guidewire Navigation for Robot-assisted Endovascular Interventions: A Knowledge-Driven Visual Guidance Approach 2024 | no | not stated | no | no |
| [14] Toward Safe Autonomous Robotic Endovascular Interventions using World Models 2026 | yes | not stated | no | no |
| [15] Learning-based catheter and guidewire-driven autonomous vascular intervention robotic system for reduced repulsive force 2022 | yes | not stated | no | no |
| [16] Reducing contact forces in the arch and supra-aortic vessels using the Magellan robot 2016 | yes | not stated | no | unclear |
| [17] Objective Assessment of Endovascular Navigation Skills with Force Sensing 2017 | yes | not stated | no | no |
| [18] Assessment of Catheter Tip Contact Force Resulting in Cardiac Perforation in Swine Atria Using Force Sensing Technology 2011 | yes | not stated | no | no |
| [19] Catheter tip force required for mechanical perforation of porcine cardiac chambers 2011 | yes | not stated | no | no |
| [20] Artificial intelligence in the autonomous navigation of endovascular interventions: a systematic review 2023 | yes | not stated | no | no |
| [21] Three-dimensional catheter tip force sensing using multi-core fiber Bragg gratings 2023 | yes | not stated | no | no |
| [22] Atherosclerotic aortic lesions increase the risk of cerebral embolism during carotid stenting in patients with complex aortic arch anatomy 2009 | no | not stated | no | no |
What I have not yet checked. Reference [3] is a 2025 PhD thesis on this exact simulator whose reward section (§ 3.4.2, “Actions and Rewards”) I could not read: the HTML render truncates before Chapter 3 and the PDF exceeded the fetch limit. Its companion paper uses a purely distance-based reward with no force term [2], which makes a hidden force-penalty sweep unlikely — but unlikely is not verified, so nothing in the gap statement rests on [3]. Reading that section is the first task on the list, and until it is done the honest answer to “have you read it?” is no. The same applies to Ng, Harada & Russell (ICML 1999) on potential-based shaping, whose result the formulation relies on: the record was confirmed, the paper itself has not been read end to end yet, and it does not get cited in the write-up until it has been.
Nothing technical in this project is claimed as new. The simulator (CathSim), the MuJoCo catheter–aorta contact-force model, the force metrics (mean force, peak force, force-thresholded safety), the idea of putting a contact-force penalty inside a reinforcement-learning reward, PPO itself, and multi-seed training are all prior art [1,2,4,7,15,17]. No new algorithm, no new environment, no new observation space, no new safety mechanism, no new metric. The only thing claimed is the controlled experiment: one coefficient varied, everything else frozen, variance reported across seeds, and the resulting safety-versus-performance relationship published as a swept curve.
Written down in advance, with the answer, because a gap statement that has not been attacked is not yet a gap statement.
Agreed, and the paragraph concedes it in its first sentence. [7] already puts guidewire tip force in the reward; [15] already does it on physical hardware; CathSim itself uses the term “reward shaping” for its distance term [1]. The claim is not about the mechanism but about its magnitude.
It says the weights were adjusted iteratively, but reports no swept values, no per-value results and no sensitivity analysis — only a final coefficient and threshold. Its ablation compares reward families, i.e. the force term present versus absent at a fixed weight. So the claim is about the published record, not about what the authors privately tried, and it has to be said that way.
Correct — [2] already used five seeds on this codebase. Seeds are what make the curve believable, not what makes it interesting. They are listed as the floor of the method, never as the finding.
And it is not claimed: the wording is “none of the studies I could verify”, scoped to the table above. The closest case acknowledges a trade-off verbally between two algorithms [14] — two discrete points, not a curve swept over a control parameter. If someone produces such a plot, the gap narrows rather than disappears.
Accepted as a limit, not argued away. The claim is about relative force under coefficient variation inside one simulator. CathSim’s simulated force distribution was compared against real-robot load-cell data and was not distinguishable [1], and animal perforation studies give order-of-magnitude anchors [18,19] — but those are cardiac ablation-catheter forces, and they are not transplanted into CathSim’s units as a threshold.
The coefficient is the treatment, not a nuisance parameter being optimised away, and the output is the exchange rate rather than a recommended setting. The contrast is [6], which sweeps learning rate and network size while holding the reward completely fixed — the one dimension this study varies is the one that benchmark did not touch.
Every one was located and read on a fetched record before it was allowed onto this page; nothing here is cited from memory. Where a record could only be verified at abstract level, or is a preprint rather than an accepted paper, the entry says so. References [20]–[22] were checked but are not cited in the paragraph above. Generated from research/lit-verified.json, so the list and the table cannot disagree.
Stated up front, before results exist, so the study is judged on its design rather than its outcome.
Does adding a filtered guidewire–vessel-wall contact-force penalty to CathSim's navigation reward reduce cumulative and peak simulated contact force during Type-I aortic arch navigation, without substantially reducing PPO task success?
A carefully scaled force penalty may reduce unnecessary vessel-wall interaction while preserving a useful level of navigation success. However, a penalty that is too strong may cause the agent to move too cautiously, fail to complete the task, or exploit the reward in unintended ways. The claim under test is therefore not “the penalty helps” but the shape of the exchange rate: how much success is given up per unit of force removed, as λ rises from zero to the point where the policy stops navigating — measured over five seeds per weight, because a difference smaller than the seed-to-seed spread is not a difference.
Recording these in advance protects the study from accidental cherry-picking once results appear.
Five weights, {0, 0.1λ*, λ*, 10λ*, 100λ*}, where λ* is measured as the ratio of task-reward magnitude to penalty magnitude over baseline episodes. Committed before the first full run. One pre-registered refinement is permitted, with a numeric trigger; no other extension.
Five independent seeds per condition — 25 training runs. The seed list is published, and every stochastic component is seeded from it. No seed is discarded for producing an inconvenient result.
An identical training-step budget for every run, set from the pilot, and the same hardware class across conditions. No condition gets extra training because it looked promising.
100 held-out episodes on fixed evaluation seeds, identical for every policy, run on the final checkpoint — never the best-looking one. Force is analysed over all episodes, not only the successful ones.
Primary outcome: the force–time integral. Primary analysis: an exact trend test across the ordered grid, blocked by seed — two pre-registered tests, Holm-corrected, critical values written down before run 1. Everything else is exploratory and labelled so.
What counts as a degenerate policy — near-zero movement, oscillation without progress, collapse in success — is defined before training, and no undocumented reward tuning happens after final results are seen.
A force reduction will not be counted as an improvement if it is accompanied by a substantial collapse in task success, near-zero movement, or oscillation without progress.
What would falsify the hypothesis, stated before running. If contact force does not fall as λ increases; or if the seed-to-seed spread inside a condition is as large as the difference between conditions; or if every λ that reduces force also destroys task success — then the hypothesis is not supported. Each of those outcomes is publishable here as a negative result, and the negative result is what gets written up.
Contact information from a physics engine can include several kinds of interaction, and the accessor CathSim ships returns only the contact-normal component rather than the full magnitude its own papers define. Before force is used as a metric or priced as a penalty, the signal is replaced, validated as specifically guidewire-to-vessel-wall contact, and shown to have something to act on.
Which MuJoCo geometry pairs count as guidewire–vessel-wall contact.
Which interactions are ignored — non-vessel objects and irrelevant simulator components.
How candidate force values are recorded at every environment step.
Low-, controlled- and high-contact trajectories used to check the signal changes plausibly.
The threshold τ is measured, not copied — and this decision is pre-registered. If at least 1% of baseline steps exceed the literature’s 2 N safety threshold in this build, τ is set to 2 N so the numbers stay comparable to published work. Otherwise τ becomes the measured upper-quartile of non-zero contact force, and the write-up states plainly that the 2 N threshold is inactive here and that no comparison to published safety percentages is valid. Which branch was taken is disclosed either way. The reason for the rule is in the experiment plan: the two CathSim papers report force scales differing by about twenty times, so copying a threshold could silently make the penalty identically zero.
The problem is defined as an MDP ⟨𝒮, 𝒜, 𝒫, ℛ, γ⟩ with an explicit, filtered reward function.
pt ∈ ℝ³qtFtdt = ‖pt − g‖atrans ∈ [−1, 1]arot ∈ [−1, 1]st+1 ~ 𝒫(· | st, at) given by MuJoCoπθ(a | s), MLP actor–criticmaxθ 𝔼[Σ γt Rt]τ, Fcap and the λ grid are outputs of the calibration pilot, committed before the first full run and not changed afterwards. No value on this page is a result.
Navigating a curved arch requires wall contact — the guidewire is steered by pressing against the vessel. A penalty on all contact therefore taxes the legitimate part of the task, so “do nothing” becomes optimal at a much lower λ and the interesting part of the curve collapses. The hinge prices only the excess above τ.
The excess is priced linearly, not quadratically. A quadratic hinge asserts that damage grows faster than load — an assertion this rigid-contact simulator cannot validate, and one that would confound how much force with how concentrated it was. Linear is the minimal form and keeps λ interpretable; a quadratic variant is pre-registered as an exploratory robustness check at one weight only.
The term is also, deliberately, not potential-based. Potential-based shaping of the form γΦ(s′) − Φ(s) is known to leave the optimal policy unchanged; a study whose whole purpose is to change which policy is optimal must therefore use a term that is provably not policy-invariant. That is the precise answer to “isn’t this just reward shaping?”
Degenerate-policy rule. A force reduction is not counted as an improvement if it comes with a collapse in success rate, near-zero movement, or oscillation without progress.
Fixed in advance and shared by all five conditions, so the penalty weight remains the only intended difference. Library defaults are used deliberately: any hand-tuning would have to happen at some particular λ, and would then silently advantage that condition.
| Parameter | Planned value | Note |
|---|---|---|
| Algorithm | PPO (clipped) | Stable-Baselines3 / CleanRL |
| Discount γ | 0.99 | Standard for continuous control |
| GAE λGAE | 0.95 | Advantage estimation |
| Clip range | 0.2 | PPO surrogate clipping |
| Learning rate | 3 × 10⁻⁴ | Adam |
| Rollout / batch | 2048 / 64 | Steps per update / minibatch |
| Policy network | MLP 2 × 64 | Stable-Baselines3 default, untuned by design |
| Penalty weight λ | {0, 0.1λ*, λ*, 10λ*, 100λ*} | The manipulated variable; λ* is measured, not chosen |
| Threshold τ | measured (decision rule) | 2 N if that threshold is live in this build, else Q75 of non-zero force |
| Random seeds | 0, 1, 2, 3, 4 | The same five in every condition (paired design) |
| Training steps | 600,000 per run | The figure both CathSim papers use; confirmed or reduced by the pilot, equally for every condition |
| Evaluation episodes | 100 per policy | Fixed held-out evaluation seeds, identical for all 25 policies |
| Reward normalisation | off | Return magnitude changes with λ; normalising it would confound the one variable |
Observation → policy → simulator → contact logging → reward → update, plus the sim-to-real boundary.
The loop repeats for millions of steps. Each step, the reward and the next observation flow back to the policy, which gradually updates πθ to earn more reward — and, for the Vascora agent, to avoid the penalty.
Vascora trains only inside CathSim. The point of this diagram is that the learned policy is deliberately isolated from hardware-specific code, so the simulator could later be replaced by a robot interface without rewriting the policy. No hardware work has been done.
Stages 05 and 06 are design intent, not implemented work. Nothing in this project has been run on physical hardware, and no sim-to-real transfer has been attempted or validated.
Select an anatomy to see the reward function it would use, and select a term to see the code line it maps to and where it acts.
These trajectories are illustrations of the reward design, not outputs of a trained policy. They show what each reward term is intended to encourage. Once agents are trained, this panel will be replaced with recorded rollouts from real checkpoints.
Below τ the penalty is exactly zero: incidental contact costs the agent nothing, so the policy is not pushed toward timidity. Above τ the excess is normalised and squared, so the marginal cost of pressing harder grows — a light brush is nearly free, a sustained push is expensive.
The shape is fixed by design; the scale (τ and λ) is not. Both are set from the measured force distribution during environment validation, then frozen before the comparison runs so they cannot be tuned to flatter the result.
Five penalty weights × five random seeds, one calibration pilot before anything long is run, and the analysis fixed in advance. A single run at a single weight is not a result.
One penalty weight and one training run is not a result. This plan fixes five weights × five seeds — 25 training runs plus a replication run — and treats the resulting trade-off curve, not any single comparison, as the finding. Nothing long runs until the instrumentation gates and the calibration pilot below have passed.
Same CathSim commit, phantom, target branch, start conditions and termination rules in every run.
PPO at Stable-Baselines3 defaults, identical network and hyperparameters in all 25 runs. Defaults are used precisely because tuning would have to happen at some λ.
Identical training-step budget, and the same five seeds in every condition — a paired design, so seed effects are blocked rather than confounded with λ.
Only λ, the weight on the clipped force hinge −λ·φ(F). The task reward is left exactly as CathSim ships it.
This is the finding that rewrote the plan, and it comes from reading the two CathSim papers side by side rather than trusting either one.
| Reported quantity | Journal version (IEEE T-MRB) | 2024 book-chapter version |
|---|---|---|
| Success radius δ | 8 mm | 4 mm |
| Episode cap | 2,000 steps | 300 steps |
| Action space | 21-dimensional | 2-dimensional |
| Contact force reported | mean 0.011 N, peak 0.121 N | 1.0 – 3.6 N |
| Safety threshold | not defined | 2 N |
| Random seeds | not stated | 5 |
The silent failure this avoids. Copy the published 2 N threshold into τ, and if the installed build behaves like the journal version — peak force around 0.12 N — then φ(F) is zero at every step, λ multiplies zero, and all five conditions become the same experiment. The output would be a perfectly flat “trade-off curve”, produced at full compute cost, with no error message anywhere. So τ is derived from this build’s own measured force distribution, and a gate kills the sweep before it starts if the penalty has nothing to act on.
The force accessor CathSim ships returns only the contact-normal component — not the full magnitude both papers define — applies no geometry filter, and gates contacts by a proximity heuristic. That signal is about to become the manipulated variable of the whole study, so it is replaced and checked first. This phase produces no science; it produces the right to run the science.
Three short baseline runs on pilot seeds that are disjoint from both the training and the evaluation seeds. That disjointness is what stops the grid from being tuned on the data it will later be judged against.
λ* is the weight at which the penalty and the task reward contribute equal magnitude over a typical baseline episode. It is a measured ratio, not a preference — and that sentence is the whole answer to “why that coefficient”. It is a calibration anchor measured at baseline, not a claim that the two terms stay balanced during training.
| Condition | λ | Role | Prediction, written before running |
|---|---|---|---|
| lam0 | 0 | Baseline — penalty absent from the code path | Anchors every contrast; highest force. Degenerate endpoint 1. |
| lam0p1x | 0.1 λ* | Negligible | Behaviour barely changes — locates where the effect starts. |
| lam1x | λ* | Comparable | The a-priori interesting setting, and precisely the one that must not be hand-picked. |
| lam10x | 10 λ* | Dominant | Substantial force reduction at real cost to success — the informative part of the curve. |
| lam100x | 100 λ* | Deliberately degenerate | Degenerate endpoint 2: the penalty should overwhelm the task. Included because it is expected to fail. |
The ceiling test asks the right question. The obvious check — “at a huge λ the agent should refuse to move” — is wrong here. Reaching the goal pays a bonus and ends the episode, which also stops the penalty accruing; and if the wire starts touching the wall, the cheapest degenerate policy is to retract into free space, not to freeze. A displacement-based gate would flag that correct behaviour as a bug. So the gate tests what is actually being asserted: at 100λ* the excess-force integral must collapse below 5% of baseline and success must fall to roughly zero. If it does not, the penalty is not in the objective as written — halt and debug, do not report a sweep.
Seeds 0–4 set the policy initialisation, the environment reset stream, the action sampling and the framework globals, recorded per run. Pilot seeds (901–903) and evaluation seeds (10000–10099) are disjoint from them and from each other. A crashed run is re-run at the same seed and both attempts are logged; a degenerate run is a data point about λ, not an outlier to remove.
The final checkpoint at exactly the shared budget — never the best-looking one — on 100 fixed episodes identical for all policies, deterministic as the primary column and stochastic as a pre-registered robustness column. At 30 episodes a success rate is worth ±17 percentage points, which would swamp the seed-to-seed spread the study is actually measuring; 100 brings it to ±9.6.
2,500 is a description of effort, not a sample size. Every inferential test operates on the 25 run-level numbers, never on the 2,500 — treating episodes as independent samples would overstate precision by about tenfold, and it is the most common way a study like this inflates itself.
A pairwise test at five seeds is arithmetically incapable of reaching p < 0.05: an exact paired test on n pairs has a floor of 2/2n, which is 0.0625 at n = 5. Reporting one anyway, without saying that, is how underpowered studies mislead. The primary analysis is therefore not pairwise.
Every row below is a publishable outcome. The write-up is drafted with the results section deliberately blank; nothing in this design requires the penalty to work.
| Hypothesis | Falsified if | What gets published |
|---|---|---|
| Force responds monotonically to λ | The trend test on the force integral fails its corrected critical value | “Not supported” — and that is the headline |
| Some λ buys real safety cheaply | No grid point cuts the force integral by ≥25% while costing ≤10 points of success | “A contact-force penalty buys no safety here that is not paid for in success.” That is a result |
| Force falls because loading falls, not because motion falls | Force drops but force-per-unit-path does not, or commanded travel collapses | The penalty produced timidity, not gentleness — a legitimate negative finding |
| The effect exceeds noise | The spread between conditions is smaller than the spread between seeds within a condition | Under-powered: report the spread, claim nothing, state the seed count that would be needed |
| The penalty is actually wired in | At 100λ* the excess-force integral does not collapse | Nothing. This is a kill-switch, not a hypothesis: halt and debug |
No wall-clock figure for this project’s hardware appears anywhere in the plan, because none can be known before the pilot measures throughput on the actual machine. The literature anchor is a range, not a number: the journal version reports about an hour for PPO at 600,000 steps, and separately reports a frame rate implying two — the two statements are inconsistent, which is exactly why the figure gets measured rather than borrowed.
Target dates, not promises: everything after the pilot depends on the throughput the pilot measures. Slippage is recorded in the log and absorbed by cutting training steps equally — never by cutting seeds, never by dropping the baseline.
| Milestone | Target | Done when | Stop condition |
|---|---|---|---|
| Gap paragraph and this plan sent for review | 2026-08-22 | Reviewer has both | Gate: no long run starts before the reply |
| Repository public, environment pinned, force accessor replaced | 2026-08-26 | A fresh clone reproduces a short run and passes its tests | Tests cannot pass ⇒ fix before anything else |
| Force validation and instrumentation report | 2026-08-29 | All checks pass or their failures are documented | Zero or monotonicity check fails ⇒ everything stops |
| Pilot, τ branch chosen, λ* measured, grid frozen and tagged | 2026-09-02 | Pilot table filled with measured numbers; the five λ committed before any sweep run | Viability gate fails ⇒ the sweep is void; re-derive τ |
| The 25 runs plus the replication run | 2026-09-16 | 26 run directories with config, manifest, curves, checkpoints and checksums | Any run not descended from the frozen tag is invalid and re-run |
| Evaluation and figures | 2026-09-19 | Every figure regenerates from a clean checkout by one command | A figure needing a manual step is a broken figure |
| Write-up published here | 2026-09-23 | Results filled with measured numbers — including a negative result, if that is what the sweep says | — |
No force number is ever published without success rate beside it. Force alone is trivially minimised by not moving, which is exactly why the result is a two-dimensional curve rather than a table of force values.
These are the quantities the study reports, shown as an empty instrument panel so the evaluation is defined before any number exists to fill it.
No performance claims are made before the full sweep is complete. Every table and figure on this page is a specification, not a finding: the axes, the row labels and the statistics are fixed now so that the numbers cannot be chosen later to flatter the result. A single number from a single run will never be published here.
The finding of this study is not a single comparison. It is the exchange rate between contact force and navigation success as the penalty weight rises — how much success is given up per unit of force removed, and where that exchange stops being worth making.
The λ values are fixed by the calibration pilot in the experiment plan, committed before the first full run. Every cell will read mean ± standard deviation across the five seeds, with all per-seed values published alongside.
| Condition | Success rate | Force–time integral primary | Excess-force integral | Peak force | Force per unit path | Degenerate seeds |
|---|---|---|---|---|---|---|
| λ = 0 baseline | pending | pending | pending | pending | pending | pending |
| 0.1 λ* | pending | pending | pending | pending | pending | pending |
| λ* | pending | pending | pending | pending | pending | pending |
| 10 λ* | pending | pending | pending | pending | pending | pending |
| 100 λ* ceiling | pending | pending | pending | pending | pending | pending |
The primary figure above, once the runs are evaluated: force against success, one marker per λ, every seed visible behind it.
λ on the x-axis, one thin line per seed across all five conditions — the visual form of the trend test, showing whether the pattern holds within a seed or only in the average.
Force and success against λ on a log axis, with λ = 0 drawn as a detached point behind an explicit axis break rather than faked as a small number.
One panel per λ, one thin line per seed, all runs shown including the bad ones.
The full per-step force distribution per λ with τ marked — did the penalty shift the whole distribution, or only clip the part it was paid to clip?
Task reward and penalty against λ. This figure exists to prove the grid really spans negligible to dominant; if the two never cross, the grid was mis-centred and the study says so.
Every figure is produced by a single command from the committed run data. A figure that cannot be regenerated from a clean checkout does not go in the write-up.
A force penalty can be “satisfied” in ways that are useless or dishonest. These failure modes are defined in advance.
The agent barely advances, keeping force near zero by never engaging the vessel.
The tip advances and retracts repeatedly without net progress toward the target.
The guidewire enters the LSA or LCC instead of continuing to the BCA target.
The agent reaches the target but maintains sustained sub-threshold contact along the wall.
Navigation is smooth overall but produces one hard impact at the curvature apex.
The method works on some seeds and fails on others.
Each confirmed failure mode will be published with a recorded rollout and a written explanation — including cases where Vascora performs worse than the baseline. Showing failures is what separates a research record from a demo reel.
Every one of these modes can produce an attractive-looking force metric while representing a policy nobody would want. F1 and F2 minimise contact by refusing to do the task. F4 hides steady loading beneath the threshold. F5 improves the average while leaving the single most dangerous moment untouched. Naming them in advance means the evaluation cannot quietly reward the wrong thing, and a reviewer can check that the analysis actually looked for them.
Stated plainly, because the value of the work depends on the reader being able to trust its boundaries.
Real vessels deform under load, which changes both the contact geometry and the force that a given motion produces. Modelling the wall as rigid means the simulated force signal is a proxy for the quantity of clinical interest, not the quantity itself. That is the single largest reason results here cannot be read as evidence about real procedures — and the reason the honest framing throughout this site is “simulated contact force”, never “tissue trauma”.
Deformable-wall simulation is listed under future directions rather than treated as a solved problem.
Everything needed to repeat the study is published together, and every design decision has an owner who can defend it. Nothing is linked until it exists and contains usable documentation.
Links are deliberately inactive until the corresponding artefact exists and contains usable documentation. A dead link in a reproducibility section is worse than no link. Publishing the repository is the first milestone of the experiment plan, and it happens on the day the pilot starts — not tidied up afterwards, because a repository created after the fact has no commit history to show.
Anyone can claim they did the work. The defence is a repository whose history shows the work happening: dated commits, saved curves for every run, and a log written while the decisions were being made rather than reconstructed afterwards.
Small, dated, single-purpose commits from the first day of the pilot. Milestones tagged: pilot-complete, grid-frozen, sweep-complete. The pre-registration is committed before the runs it constrains, so its timestamp proves it came first.
Each of the 25 runs writes its own directory with the exact config, the seed, the git commit it was launched from and the environment fingerprint. A run that cannot say which code produced it is discarded and repeated.
Learning curves are saved as raw CSV plus the TensorBoard event files for all 25 runs, so a reader can re-plot them, spot a cherry-picked seed, or check a claim against the underlying numbers.
A plain-text log in the repository, one entry per working session: what was tried, what happened, what was decided and why. Entries are appended, never rewritten.
Where tools, tutorials, code or advice were used, they are named in the log and the README. Declared help is a strength; undeclared help is the thing that collapses under one interview question.
Every non-obvious choice — algorithm, penalty form, grid, seed count — has a written justification in the log at the time it was made, which is what makes it answerable months later.
| Saved per run | File | Why it matters |
|---|---|---|
| Configuration | config.yaml | Every hyperparameter, the λ value and the environment settings that produced this run. |
| Seed and provenance | run_meta.json | Seed, git commit hash, package versions, hardware, start and end time, wall-clock. |
| Training curve | progress.csv | Reward, success rate, mean and peak force against timesteps — the raw numbers behind every plot. |
| Event log | events.out.tfevents | TensorBoard record, so the run can be inspected without trusting a rendered image. |
| Evaluation | eval_episodes.json | Per-episode outcome, per-step force trace, failure classification for the fixed evaluation set. |
| Policy | final_model.zip | The final checkpoint — the one that is evaluated, not the best-looking one. |
| Console | stdout.log | Warnings and crashes that a summary table would hide. |
The log is a file in the repository. It is reproduced here in outline so a reader can see what an entry contains. Only entries that have actually happened are shown — the log is not pre-filled with future dates.
If a choice cannot be justified out loud, it is not yet a choice — it is an inherited default. These are the answers as they stand today; they change only if the pilot says something different, and the change goes in the log.
Provided for completeness while the project is in progress. This is an unpublished student research project, not a peer-reviewed paper.
@misc{aryan_vascora,
author = {Kumar Aryan},
title = {Vascora: Contact-Force-Aware Reward Shaping for
Simulated Aortic Arch Navigation in CathSim using PPO},
year = {2026},
note = {Unpublished student research project. Simulation only;
no clinical validation. Work in progress.},
howpublished = {Project website}
}
Vascora is an independent research project by Kumar Aryan.
I build rigorous, reproducible projects at the intersection of reinforcement learning, simulation, robotics and healthcare. Vascora studies how reward design influences contact-force behaviour in simulated endovascular navigation.
Listed as intent, not as completed work.
Test whether findings hold beyond a single geometry, including tortuous and aneurysmal anatomies.
Generalise beyond the BCA to other arch branches with differing approach angles.
Compare PPO against SAC or DDPG under the same matched protocol.
Explore CVaR-style objectives that price the tail of the force distribution rather than the mean.
Move beyond rigid walls toward compliant tissue — the largest single realism gap.
Characterise what transfers and what does not, carefully and without overclaiming.
Vascora is an independent student research project. It is not affiliated with, endorsed by, or a product of any medical-device manufacturer or clinical institution.