Research project · 2026
CauSyncVLA: Causal Latent World Modeling for Asynchronous Robot Control
- School of Electrical and Electronic Engineering, Nanyang Technological University
- College of Computing and Data Science, Nanyang Technological University
- Centre for Frontier AI Research (CFAR) and Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR)
* Corresponding author
Under review
Abstract
Vision–language–action policies must react while visual inference is still running. Overlapping inference and execution reduces waiting, but leaves perceptual memory stale and successive action chunks disconnected. CauSyncVLA couples a causal latent world model, which updates delayed perceptual memory from executed state–action history, with predictive flow warm-starting, which refines the unexecuted plan in one step. On LIBERO-10, it reaches 93.8% success with 5.04× the model replanning capacity of a ten-step synchronous baseline on a shared GPU.

In motion
Method
A slow visual stream publishes key–value (KV) memory anchors, while a fast control stream propagates those anchors to the current execution time and refines the action horizon. The backbone and action expert remain frozen; only the lightweight latent predictor is trained.

Causal latent world model
A predictor updates delayed KV anchors using the state transitions and commands actually executed since the anchor was captured.
Action-sensitive distillation
In addition to KV reconstruction, the frozen expert's response supervises the predictor to preserve memory features that matter for control.
Predictive flow warm-starting
The unexecuted action tail is aligned to the current state and refined in one flow step, without training the action expert again.
Execution-grounded control
At a replanning boundary, the latest visual anchor $z_a$ was captured at time $a$, while the robot is now at $t$. The history $\mathcal H_{a:t}$ contains realized state transitions and executed commands; $\mathcal Q_t$ is the unexecuted tail of the preceding plan, shifted past consumed commands and rebased to the current state. Both records therefore refer to actual execution.
The latent model predicts a residual to the delayed KV anchor; the age gate leaves a freshly captured anchor unchanged. Causality here means using information available by the decision time. It does not imply prediction of future pixels, or identification of external physical dynamics.
Predictive flow warm-starting
Instead of drawing a new trajectory from pure noise, PFWS mixes the shifted prior with Gaussian noise at flow time $\rho$, then applies one Euler step of the frozen action expert:
Smaller $\rho$ retains more of the prior trajectory, while $\rho=1$ recovers cold-start sampling. The experiments use $\rho=0.5$; no additional action-expert training is required.
Action-sensitive distillation
During training, the fresh anchor $z_t$ is a teacher for the prediction $\hat z_t$. Both query the same frozen expert with identical noise, flow time and state. KV reconstruction preserves the representation, and the action term prioritizes errors that change the expert's output:
Here $\hat A$ and $A^\star$ are the student and detached teacher outputs; $m$ masks valid KV entries and $w$ weights action coordinates. Only the latent predictor is updated.
Evaluation
The controller builds on a frozen $\pi_{0.5}$[1] backbone. We evaluate 500 official starts on LIBERO-10[2], 2,519 paired perturbation variants on LIBERO-Plus[3], and 100 shared starts under a 20 Hz, 26-second closed-loop budget. Physical evaluation uses a Franka FR3 arm with two cameras on cup stacking, held-out fruit placement, and vegetable sorting. Model-only throughput and timed closed-loop response measure different aspects of performance. The 24.27M-parameter predictor is trained for 12,000 updates; simulation uses horizon 10 and stride 5, while the FR3 experiments use horizon 10 and stride 2 with live 30 Hz perception. Model capacity excludes simulator and robot execution.
Results
CauSyncVLA reaches 93.8% native success while reducing mean batch-one model latency from 60.01 ms to 11.91 ms against the ten-step synchronous baseline. Under the separate timed closed-loop protocol, mean per-episode median observation-to-first-action response drops from 176.73 ms to 83.04 ms. PFWS improves continuity, but its benefits under perturbation depend on the type of shift: large viewpoint or layout changes can favor cold starts.
| Controller | Decoder | H/C | Success (%) ↑ | Mean (ms) ↓ | P95 (ms) ↓ | Plans/s ↑ | Actions/s ↑ |
|---|---|---|---|---|---|---|---|
| $\pi_{0.5}$ | 10-step flow | 10/5 | 91.20 | 60.01 | 60.20 | 16.66 | 83.32 |
| $\pi_{0.5}$ | 1-step flow | 10/5 | 90.80 | 36.00 | 36.32 | 27.78 | 138.88 |
| $\pi_{0.5}$ + PFWS | 1-step flow | 10/5 | 93.00 | 35.54 | 35.94 | 28.14 | 140.70 |
| GR00T N1.7[7] | 4-step flow | 16/5 | 85.80 | 59.37 | 59.90 | 16.84 | 84.21 |
| OpenVLA-OFT[8] | Parallel L1 | 8/5 | 89.20 | 62.71 | 62.91 | 15.95 | 79.73 |
| RTC[4] | 10-step + guidance | 10/5 | 91.80 | 179.82 | 182.66 | 5.56 | 27.81 |
| Latent Bridge R0[5] | 2-step + bridge | 10/5 | 91.80 | 63.06 | 78.29 | 15.86 | 79.29 |
| VLA-Rolling[6] | Adaptive 1–10 | 50/10 | 93.40 | 166.47 | 202.06 | 6.01 | 60.07 |
| CauSyncVLA | 1-step flow | 10/5 | 93.80 | 11.91 | 13.38 | 83.96 | 419.82 |
Model capacity and closed-loop response
Reducing the synchronous expert from ten flow steps to one lowers mean latency from 60.01 ms to 36.00 ms. Moving visual encoding off the fast path reduces the shared-GPU measurement further to 11.91 ms. The resulting 5.04× is model replanning capacity, not a measured fivefold increase in robot dispatch. Under the separate 20 Hz timed protocol, measured replans rise from 2.57 to 3.28 per second and completion rises from 86% to 91% across 100 shared starts.

| Controller | Success (%) ↑ | Response P50 / P95 (ms) ↓ | Replans/s ↑ | Actions/s ↑ |
|---|---|---|---|---|
| $\pi_{0.5}$ | 86.00 | 176.73 / 189.97 | 2.57 | 13.02 |
| Latent Bridge R0 | 82.00 | 183.88 / 189.79 | 2.48 | 12.58 |
| VLA-Rolling | 87.00 | 295.21 / 345.94 | 1.28 | 13.20 |
| CauSyncVLA | 91.00 | 83.04 / 90.53 | 3.28 | 16.60 |
Does execution history preserve useful memory?
On 320 held-out windows, stale anchors yield action MSE 0.10355 against fresh-anchor expert outputs. Conditioning on ordered execution history reduces it to 0.00983. Age and endpoints help, but lose intermediate motion; shuffling state–action pairs retains much of the geometric signal while degrading the ordered model. These are prediction errors within the frozen expert, not a direct test of physical dynamics.

| History conditioning | KV error ↓ | Action MSE ↓ |
|---|---|---|
| Stale anchor (no update) | 0.42430 | 0.10355 |
| Age only | 0.37217 | 0.07221 |
| Age + endpoint states | 0.35669 | 0.04602 |
| Shuffled history | 0.32037 | 0.01107 |
| Full causal history | 0.31958 | 0.00983 |
Why warm-start the action trajectory?
With fresh synchronous vision, PFWS lowers episode-averaged boundary flips from 2.57% to 1.36% over 500 paired starts, and boundary excess from +0.88 to −0.03 percentage points. The paper finds its clearest contribution in continuity; the paired evidence for native success gains is inconclusive. In a shared PCA projection, warm-started paths begin closer to the endpoint, reducing one-step endpoint error by 19.7% without changing the expert.

Robustness and component ablations
Across 2,519 LIBERO-Plus variants, PFWS increases overall success from 76.38% to 78.09% within CauSyncVLA, but it does not help every shift. Noise and robot-state changes benefit from trajectory reuse; viewpoint and layout changes can favor cold starts. The synchronous PFWS reference achieves 80.15% overall, so these results do not establish unconditional robustness over fresh vision.

In matched module ablations, removing the world model lowers success to 80.20%, while removing PFWS increases pooled flips to 3.69%. The full matched control reaches 93.40% with 1.86% flips. In a separate matched objective comparison, adding action distillation reduces action MSE from 0.02018 to 0.00983 and raises success from 85.60% to 92.80%, despite a higher KV error. This supports supervising control-relevant memory directions, not just minimizing uniform KV distance. These matched cohorts differ from the 93.80% main-result run.
| Controller | Success (%) ↑ | Pooled flips (%) ↓ |
|---|---|---|
| Without latent world model | 80.20 | 2.30 |
| Without PFWS | 92.00 | 3.69 |
| Synchronous PFWS (fresh KV) | 92.00 | 1.26 |
| Full model | 93.40 | 1.86 |
| Objective | Success (%) ↑ | KV error ↓ | Action MSE ↓ |
|---|---|---|---|
| Without action distillation | 85.60 | 0.28461 | 0.02018 |
| With action distillation | 92.80 | 0.31958 | 0.00983 |
Physical robot evaluation
On the physical robot, pooled success rises from 72.46% to 86.96%, with median response falling from 72.30 ms to 13.17 ms. These are whole-system comparisons: active dispatch rates differ and both controllers pause at shared gripper barriers.
| Task | Controller | Success (%) ↑ | Active actions/s ↑ | Active plans/s ↑ | Response P50 (ms) ↓ | Response P95 (ms) ↓ | Speed CV ↓ | Accel. (m/s²) ↓ | Time (s) ↓ |
|---|---|---|---|---|---|---|---|---|---|
| Cup stacking | $\pi_{0.5}$ | 73.91 | 20.00 | 9.98 | 72.34 | 75.03 | 0.530 | 0.390 | 28.97 |
| Cup stacking | CauSyncVLA | 82.61 | 43.22 | 21.67 | 13.30 | 25.50 | 0.453 | 0.289 | 19.63 |
| Fruit placement | $\pi_{0.5}$ | 69.57 | 20.08 | 10.07 | 72.30 | 73.42 | 0.487 | 0.348 | 22.87 |
| Fruit placement | CauSyncVLA | 95.65 | 42.41 | 21.27 | 14.28 | 31.75 | 0.433 | 0.289 | 16.62 |
| Vegetable sorting | $\pi_{0.5}$ | 73.91 | 20.04 | 10.03 | 72.26 | 73.28 | 0.659 | 0.405 | 23.50 |
| Vegetable sorting | CauSyncVLA | 82.61 | 43.26 | 21.66 | 12.82 | 23.54 | 0.525 | 0.332 | 17.75 |
| All tasks | $\pi_{0.5}$ | 72.46 | 20.06 | 10.03 | 72.30 | 73.47 | 0.531 | 0.387 | 25.16 |
| All tasks | CauSyncVLA | 86.96 | 43.13 | 21.60 | 13.17 | 25.53 | 0.462 | 0.296 | 17.93 |
Grasp 1
Place 1
Grasp 2
Place 2
Grasp 1
Place 1
Grasp 2
Place 2Physical trials compare deployed systems with independently reset scenes and different active dispatch rates; both pause at shared gripper barriers. They do not isolate a fixed-frequency module effect. The experiments use one frozen $\pi_{0.5}$ backbone, and unobserved scene changes between visual anchors remain a limitation; multi-backbone physical validation and event-triggered refresh remain future work.
Citation
@unpublished{pan2026causyncvla,
title={CauSyncVLA: Causal Latent World Modeling for Asynchronous Robot Control},
author={Xu Pan and Chubin Zhang and Xingrui Yu and Runmin Lan and Zihao Wang and Wenkai Guo and Bofang Jia and Bo An and Ziwei Wang},
year={2026},
note={Under review}
}References
- K. Black, N. Brown, J. Darpinian, et al. “$\pi_{0.5}$: A Vision-Language-Action Model with Open-World Generalization.” CoRL, 2025.Paper
- B. Liu, Y. Zhu, C. Gao, et al. “LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning.” NeurIPS, 2023.Paper
- S. Fei, S. Wang, J. Shi, et al. “LIBERO-Plus: A Progressive Robustness Benchmark for Visual-Language-Action Models.” CVPR, 2026.Paper
- K. Black, M. Y. Galliker, and S. Levine. “Real-Time Execution of Action Chunking Flow Policies.” NeurIPS, 2025.arXiv
- Y. Liu, Y. Li, Z. Tang, et al. “Latent Bridge: Feature Delta Prediction for Efficient Dual-System Vision-Language-Action Model Inference.” arXiv:2605.02739, 2026.arXiv
- S. He, Y. Pan, Q. Yu, et al. “VLA-Rolling: Fewer Denoising Steps, Better Control via Adaptive Action Reuse.” arXiv:2610.25043, 2026.arXiv
- NVIDIA. “GR00T N1.7 LIBERO: Official policy checkpoints.” 2026.Model
- M. J. Kim, C. Finn, and P. Liang. “Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success.” RSS, 2025.Paper