skip to content
Home
More Works

Research project · 2026

CauSyncVLA: Causal Latent World Modeling for Asynchronous Robot Control

  1. School of Electrical and Electronic Engineering, Nanyang Technological University
  2. College of Computing and Data Science, Nanyang Technological University
  3. Centre for Frontier AI Research (CFAR) and Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR)

* Corresponding author

Under review

Vision–language–action policies must react while visual inference is still running. Overlapping inference and execution reduces waiting, but leaves perceptual memory stale and successive action chunks disconnected. CauSyncVLA couples a causal latent world model, which updates delayed perceptual memory from executed state–action history, with predictive flow warm-starting, which refines the unexecuted plan in one step. On LIBERO-10, it reaches 93.8% success with 5.04× the model replanning capacity of a ten-step synchronous baseline on a shared GPU.

Synchronous and naive asynchronous control compared with CauSyncVLA's coupled perceptual-memory prediction and action correction
Fig. 1. CauSyncVLA aligns fast feedback control with slower visual updates, using execution history to predict perceptual memory and an unexecuted plan to warm-start trajectory correction.
Real-robot rollouts compare synchronous $\pi_{0.5}$ and CauSyncVLA on three-cup stacking, fruit placement, and vegetable sorting. The trials have independently reset scenes.

A slow visual stream publishes key–value (KV) memory anchors, while a fast control stream propagates those anchors to the current execution time and refines the action horizon. The backbone and action expert remain frozen; only the lightweight latent predictor is trained.

CauSyncVLA architecture showing the causal latent world model, action-sensitive distillation, and predictive flow warm-starting
Fig. 2. Architecture and training: executed state–action history conditions KV-memory prediction, and a shifted, rebased action tail initializes single-step flow refinement.
01

Causal latent world model

A predictor updates delayed KV anchors using the state transitions and commands actually executed since the anchor was captured.

02

Action-sensitive distillation

In addition to KV reconstruction, the frozen expert's response supervises the predictor to preserve memory features that matter for control.

03

Predictive flow warm-starting

The unexecuted action tail is aligned to the current state and refined in one flow step, without training the action expert again.

Execution-grounded control

At a replanning boundary, the latest visual anchor $z_a$ was captured at time $a$, while the robot is now at $t$. The history $\mathcal H_{a:t}$ contains realized state transitions and executed commands; $\mathcal Q_t$ is the unexecuted tail of the preceding plan, shifted past consumed commands and rebased to the current state. Both records therefore refer to actual execution.

$$A_t=\mathcal R_\psi\!\left(\mathcal Q_t,W_\theta(z_a,\mathcal H_{a:t})\right),\qquad \hat z_t=z_a+\mathbb I[k>0]\,\delta z_t.$$

The latent model predicts a residual to the delayed KV anchor; the age gate leaves a freshly captured anchor unchanged. Causality here means using information available by the decision time. It does not imply prediction of future pixels, or identification of external physical dynamics.

Predictive flow warm-starting

Instead of drawing a new trajectory from pure noise, PFWS mixes the shifted prior with Gaussian noise at flow time $\rho$, then applies one Euler step of the frozen action expert:

$$\begin{aligned}x_\rho&=(1-\rho)\mathcal Q_t+\rho\epsilon,\quad \epsilon\sim\mathcal N(0,I),\\ A_t&=x_\rho-\rho\,v_\psi(x_\rho,\rho\mid\hat z_t).\end{aligned}$$

Smaller $\rho$ retains more of the prior trajectory, while $\rho=1$ recovers cold-start sampling. The experiments use $\rho=0.5$; no additional action-expert training is required.

Action-sensitive distillation

During training, the fresh anchor $z_t$ is a teacher for the prediction $\hat z_t$. Both query the same frozen expert with identical noise, flow time and state. KV reconstruction preserves the representation, and the action term prioritizes errors that change the expert's output:

$$\mathcal L(\theta)=\underbrace{\left\|\frac{\hat z_t-z_t}{\sigma^\Delta}\right\|_m^2}_{\mathcal L_{\mathrm{KV}}}+\lambda\underbrace{\left\|\hat A-A^\star\right\|_w^2}_{\mathcal L_{\mathrm{act}}}.$$

Here $\hat A$ and $A^\star$ are the student and detached teacher outputs; $m$ masks valid KV entries and $w$ weights action coordinates. Only the latent predictor is updated.

The controller builds on a frozen $\pi_{0.5}$[1] backbone. We evaluate 500 official starts on LIBERO-10[2], 2,519 paired perturbation variants on LIBERO-Plus[3], and 100 shared starts under a 20 Hz, 26-second closed-loop budget. Physical evaluation uses a Franka FR3 arm with two cameras on cup stacking, held-out fruit placement, and vegetable sorting. Model-only throughput and timed closed-loop response measure different aspects of performance. The 24.27M-parameter predictor is trained for 12,000 updates; simulation uses horizon 10 and stride 5, while the FR3 experiments use horizon 10 and stride 2 with live 30 Hz perception. Model capacity excludes simulator and robot execution.

93.8%LIBERO-10 success across 500 starts
5.04×shared-GPU model capacity vs. synchronous baseline
86.96%pooled real-robot success across 69 trials

CauSyncVLA reaches 93.8% native success while reducing mean batch-one model latency from 60.01 ms to 11.91 ms against the ten-step synchronous baseline. Under the separate timed closed-loop protocol, mean per-episode median observation-to-first-action response drops from 176.73 ms to 83.04 ms. PFWS improves continuity, but its benefits under perturbation depend on the type of shift: large viewpoint or layout changes can favor cold starts.

ControllerDecoderH/CSuccess (%) ↑Mean (ms) ↓P95 (ms) ↓Plans/s ↑Actions/s ↑
$\pi_{0.5}$10-step flow10/591.2060.0160.2016.6683.32
$\pi_{0.5}$1-step flow10/590.8036.0036.3227.78138.88
$\pi_{0.5}$ + PFWS1-step flow10/593.0035.5435.9428.14140.70
GR00T N1.7[7]4-step flow16/585.8059.3759.9016.8484.21
OpenVLA-OFT[8]Parallel L18/589.2062.7162.9115.9579.73
RTC[4]10-step + guidance10/591.80179.82182.665.5627.81
Latent Bridge R0[5]2-step + bridge10/591.8063.0678.2915.8679.29
VLA-Rolling[6]Adaptive 1–1050/1093.40166.47202.066.0160.07
CauSyncVLA1-step flow10/593.8011.9113.3883.96419.82
Table 1. LIBERO-10 task competence and batch-one model inference capacity. Success uses 500 starts and a 520-action budget; timing measures model inference rather than timed closed-loop execution. Bold and underline mark the best and second-best values; H is the action horizon and C is the replanning stride.

Model capacity and closed-loop response

Reducing the synchronous expert from ten flow steps to one lowers mean latency from 60.01 ms to 36.00 ms. Moving visual encoding off the fast path reduces the shared-GPU measurement further to 11.91 ms. The resulting 5.04× is model replanning capacity, not a measured fivefold increase in robot dispatch. Under the separate 20 Hz timed protocol, measured replans rise from 2.57 to 3.28 per second and completion rises from 86% to 91% across 100 shared starts.

Normalized model replanning capacity for synchronous and asynchronous configurations
Fig. 3. Batch-one model replanning capacity normalized to ten-step synchronous$\pi_{0.5}$, at stride 5. Shared GPU runs visual encoding concurrently; isolated System 1 removes this contention.
ControllerSuccess (%) ↑Response P50 / P95 (ms) ↓Replans/s ↑Actions/s ↑
$\pi_{0.5}$86.00176.73 / 189.972.5713.02
Latent Bridge R082.00183.88 / 189.792.4812.58
VLA-Rolling87.00295.21 / 345.941.2813.20
CauSyncVLA91.0083.04 / 90.533.2816.60
Table 2. Timed execution over 100 shared starts at 20 Hz with a 26-second budget. Policy inference and rendering share one RTX 5090; response values are means of per-episode P50 and P95.

Does execution history preserve useful memory?

On 320 held-out windows, stale anchors yield action MSE 0.10355 against fresh-anchor expert outputs. Conditioning on ordered execution history reduces it to 0.00983. Age and endpoints help, but lose intermediate motion; shuffling state–action pairs retains much of the geometric signal while degrading the ordered model. These are prediction errors within the frozen expert, not a direct test of physical dynamics.

Downstream action mean squared error versus visual anchor staleness for stale and predicted memory
Fig. 4. Action error relative to fresh-anchor outputs across 320 held-out windows; the vertical axis is logarithmic. The predictor maintains lower error across sampled staleness horizons up to 45 steps.
History conditioningKV error ↓Action MSE ↓
Stale anchor (no update)0.424300.10355
Age only0.372170.07221
Age + endpoint states0.356690.04602
Shuffled history0.320370.01107
Full causal history0.319580.00983
Table 3. Execution-history ablations across 320 held-out windows. Masked or permuted inputs preserve state–action pairings; bold and underline mark the best and second-best values.

Why warm-start the action trajectory?

With fresh synchronous vision, PFWS lowers episode-averaged boundary flips from 2.57% to 1.36% over 500 paired starts, and boundary excess from +0.88 to −0.03 percentage points. The paper finds its clearest contribution in continuity; the paired evidence for native success gains is inconclusive. In a shared PCA projection, warm-started paths begin closer to the endpoint, reducing one-step endpoint error by 19.7% without changing the expert.

Projected cold and warm-started flow trajectories across eight matched seeds
Fig. 5. Frozen-expert flow paths projected into a shared PCA basis for eight matched seeds. Circles mark initial latents and squares mark dense endpoints; these are projections rather than physical robot trajectories.

Robustness and component ablations

Across 2,519 LIBERO-Plus variants, PFWS increases overall success from 76.38% to 78.09% within CauSyncVLA, but it does not help every shift. Noise and robot-state changes benefit from trajectory reuse; viewpoint and layout changes can favor cold starts. The synchronous PFWS reference achieves 80.15% overall, so these results do not establish unconditional robustness over fresh vision.

Change in success rate from predictive flow warm-starting across LIBERO-Plus perturbations
Fig. 6. PFWS success differences versus cold starts, in percentage points, across LIBERO-Plus perturbations for synchronous vision and CauSyncVLA with a fixed world model.

In matched module ablations, removing the world model lowers success to 80.20%, while removing PFWS increases pooled flips to 3.69%. The full matched control reaches 93.40% with 1.86% flips. In a separate matched objective comparison, adding action distillation reduces action MSE from 0.02018 to 0.00983 and raises success from 85.60% to 92.80%, despite a higher KV error. This supports supervising control-relevant memory directions, not just minimizing uniform KV distance. These matched cohorts differ from the 93.80% main-result run.

ControllerSuccess (%) ↑Pooled flips (%) ↓
Without latent world model80.202.30
Without PFWS92.003.69
Synchronous PFWS (fresh KV)92.001.26
Full model93.401.86
Table 4. Matched module ablations over 500 starts; pooled flip rates are distinct from episode-averaged boundary flips above.
ObjectiveSuccess (%) ↑KV error ↓Action MSE ↓
Without action distillation85.600.284610.02018
With action distillation92.800.319580.00983
Table 5. Matched 12,000-update predictors, comparing uniform KV reconstruction with action-sensitive distillation.

Physical robot evaluation

On the physical robot, pooled success rises from 72.46% to 86.96%, with median response falling from 72.30 ms to 13.17 ms. These are whole-system comparisons: active dispatch rates differ and both controllers pause at shared gripper barriers.

TaskControllerSuccess (%) ↑Active actions/s ↑Active plans/s ↑Response P50 (ms) ↓Response P95 (ms) ↓Speed CV ↓Accel. (m/s²) ↓Time (s) ↓
Cup stacking$\pi_{0.5}$73.9120.009.9872.3475.030.5300.39028.97
Cup stackingCauSyncVLA82.6143.2221.6713.3025.500.4530.28919.63
Fruit placement$\pi_{0.5}$69.5720.0810.0772.3073.420.4870.34822.87
Fruit placementCauSyncVLA95.6542.4121.2714.2831.750.4330.28916.62
Vegetable sorting$\pi_{0.5}$73.9120.0410.0372.2673.280.6590.40523.50
Vegetable sortingCauSyncVLA82.6143.2621.6612.8223.540.5250.33217.75
All tasks$\pi_{0.5}$72.4620.0610.0372.3073.470.5310.38725.16
All tasksCauSyncVLA86.9643.1321.6013.1725.530.4620.29617.93
Table 6. Franka FR3 results by task and pooled across the three task suites. Dispatch, latency, and end-effector motion are all-trial medians; time is the mean over solved trials. Active rates exclude synchronization waits.
$\pi_{0.5}$
Synchronous baseline cup stacking: Grasp 1Grasp 1
Synchronous baseline cup stacking: Place 1Place 1
Synchronous baseline cup stacking: Grasp 2Grasp 2
Synchronous baseline cup stacking: Place 2Place 2
CauSyncVLA
CauSyncVLA cup stacking: Grasp 1Grasp 1
CauSyncVLA cup stacking: Place 1Place 1
CauSyncVLA cup stacking: Grasp 2Grasp 2
CauSyncVLA cup stacking: Place 2Place 2
Fig. 7. Stage-aligned cup stacking on Franka FR3: synchronous $\pi_{0.5}$ above and CauSyncVLA below. Trails follow the visible gripper housing through four grasp–place phases. The two successful trials use independently reset object layouts; the illustrations do not establish a matched initial state or replace the quantitative results.

Physical trials compare deployed systems with independently reset scenes and different active dispatch rates; both pause at shared gripper barriers. They do not isolate a fixed-frequency module effect. The experiments use one frozen $\pi_{0.5}$ backbone, and unobserved scene changes between visual anchors remain a limitation; multi-backbone physical validation and event-triggered refresh remain future work.

@unpublished{pan2026causyncvla,
  title={CauSyncVLA: Causal Latent World Modeling for Asynchronous Robot Control},
  author={Xu Pan and Chubin Zhang and Xingrui Yu and Runmin Lan and Zihao Wang and Wenkai Guo and Bofang Jia and Bo An and Ziwei Wang},
  year={2026},
  note={Under review}
}
  1. K. Black, N. Brown, J. Darpinian, et al. “$\pi_{0.5}$: A Vision-Language-Action Model with Open-World Generalization.” CoRL, 2025.Paper
  2. B. Liu, Y. Zhu, C. Gao, et al. “LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning.” NeurIPS, 2023.Paper
  3. S. Fei, S. Wang, J. Shi, et al. “LIBERO-Plus: A Progressive Robustness Benchmark for Visual-Language-Action Models.” CVPR, 2026.Paper
  4. K. Black, M. Y. Galliker, and S. Levine. “Real-Time Execution of Action Chunking Flow Policies.” NeurIPS, 2025.arXiv
  5. Y. Liu, Y. Li, Z. Tang, et al. “Latent Bridge: Feature Delta Prediction for Efficient Dual-System Vision-Language-Action Model Inference.” arXiv:2605.02739, 2026.arXiv
  6. S. He, Y. Pan, Q. Yu, et al. “VLA-Rolling: Fewer Denoising Steps, Better Control via Adaptive Action Reuse.” arXiv:2610.25043, 2026.arXiv
  7. NVIDIA. “GR00T N1.7 LIBERO: Official policy checkpoints.” 2026.Model
  8. M. J. Kim, C. Finn, and P. Liang. “Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success.” RSS, 2025.Paper