verl-vla v0.1.0: A Unified Cloud-Edge Post-Training Loop for Large-Scale VLA

As large-scale vision-language-action (VLA) models such as Pi0.5 advance, robot policies are gaining stronger multimodal understanding and cross-task generalization. Turning those capabilities into reliable task performance, however, still requires continuous cycles of data collection, model fine-tuning, policy evaluation, and reinforcement learning post-training. Organizing these stages into a sustainable, iterative loop is a central challenge for bringing large-scale VLAs into practice.

verl-vla is an open-source VLA post-training framework built on verl. It addresses this challenge by bringing physical robots, simulated environments, teleoperation devices, and compute resources into one unified loop. This post introduces its architecture and shows how distributed training clusters, web-based data collection, and composable workflows connect data collection, fine-tuning, evaluation, and reinforcement learning, while reproducible recipes accelerate VLA deployment.

Why Large-Scale VLA Post-Training Needs a Cloud-Edge System

Robot learning has traditionally revolved around local workflows: developers collect demonstrations beside a robot arm, train policies on a local workstation, and deploy the resulting model back to the robot. For policies of manageable scale, such as ACT and Diffusion Policy, the robot, data, model, and GPUs typically reside in the same environment, enabling rapid iteration across data collection, imitation learning, deployment, and human intervention.

As models scale to large VLAs, however, this local loop begins to encounter new constraints. Model inference and fine-tuning often depend on cloud GPUs or multi-node clusters; high-fidelity simulators may run on dedicated servers; and physical robots and teleoperation devices remain at the edge. Post-training must also accommodate different environments, input devices, and training paradigms, including supervised fine-tuning, retraining on corrective data, and offline or online reinforcement learning. Rebuilding the workflow whenever any one of these elements changes quickly makes the engineering cost exceed the algorithmic cost.

Large-scale VLA post-training therefore needs more than a collection of disconnected tools. It needs a unified cloud-edge system that brings data collection, policy optimization, and evaluation into the same loop. The figure below shows the unified architecture that verl-vla builds toward this goal.

TrainCluster: A Unified Training Cluster Across Cloud, Edge, and Heterogeneous Resources

As described above, distributed resources are the first challenge in cloud-edge VLA post-training. Models, environments, training jobs, and human inputs no longer share a single runtime, so deployment, scheduling, and coordination become concerns that every workflow must address. Solving this problem begins with bringing those distributed resources into one system. TrainCluster is the execution foundation that verl-vla provides for that purpose.

As Figure 1 shows, TrainCluster brings training, rollout, environment interaction, evaluation, data recording, and checkpoint management behind a unified interface. Higher-level workflows only describe the operations required at each stage; they do not need to manage Ray, resource pools, or simulator processes directly. TrainCluster launches the required workers from the cluster configuration, schedules resources, and manages their lifecycles. Resources are organized by role rather than location: environment workers can run on edge devices or simulator nodes, rollout workers can share a machine with training or run on dedicated GPUs, and actor workers can scale across multiple GPUs and nodes. Data collection, evaluation, and online reinforcement learning workflows simply compose the roles they need.

APIs such as start(), record(), rollout(), train(), eval(), and shutdown() expose these capabilities consistently. In disaggregated training-and-serving deployments, TrainCluster also synchronizes weights between the training and rollout models. Developers can validate a workflow on a single machine first, then distribute environments, rollout, and training across nodes as needed, without reimplementing post-training logic for each deployment topology.

Web-Based Data Collection and Human Intervention for Distributed Cloud-Edge Deployments

TrainCluster brings distributed compute and environment resources into one system, but human-in-the-loop data collection still faces a separation between devices and environments: an operator’s keyboard, gamepad, or XR controller is connected locally, while the environment may run in a cloud simulator or on an edge-side robot. Traditional teleoperation binds an input device directly to an environment process, which does not adapt well to this cross-device, cross-network setting.

To address this, verl-vla starts a web service alongside the environment. Operators can open the teleoperation interface from any device and connect their own local input hardware. Whether the environment is LIBERO or Isaac Lab-Arena in the cloud, or a robot at the edge, operators use the same interface to observe state, send control commands, and record trajectories. Teleoperation adapters map device signals to environment-specific actions, so adding a new environment only requires adapter logic rather than rebuilding device connections, web visualization, or the data-collection path. Figure 2 shows this unified web interface across different environments.

DAgger also needs a different interaction pattern in cloud-edge deployments. Smaller local models can infer at high frequency, allowing teleoperation actions to replace model actions step by step. Large VLAs, however, face inference and cloud-edge communication latency, so real deployments usually rely on action chunks to keep edge-side execution smooth. verl-vla therefore switches control at the trajectory level: as Figure 3 shows, an operator can interrupt autonomous execution at any time, insert a recovery or correction trajectory of arbitrary length, and then return control to the policy. The complete trajectory and its control state are recorded together and passed into subsequent fine-tuning and reinforcement learning workflows.

Orchestrating Complete Post-Training Workflows

TrainCluster unifies cloud-edge resources, and the web environment loop unifies data collection and human intervention. Workflow then orchestrates these capabilities into complete post-training procedures. It defines the execution order and data flow across stages, while TrainCluster handles resource scheduling and worker lifecycles and Trainers focus on algorithmic updates. This separation decouples workflow orchestration, distributed execution, and algorithm implementation.

The RECAP recipe illustrates this design. It reuses existing evaluation, trajectory collection, SFT training, and rollout capabilities to perform evaluation, data collection, return computation, value-model training, advantage annotation, and policy updates in sequence. The RECAP-specific pieces are limited to data processing for returns, advantages, and the value model; existing components provide the rest. Workflow connects these stages into an executable, recoverable iteration process.

Once a model and environment have been integrated, developers can pass data, models, and checkpoints between data collection, fine-tuning, evaluation, and different reinforcement learning workflows, then compose new post-training procedures as needed. Implementing a new algorithm like RECAP only requires its algorithm-specific data flow and a small amount of logic, without reintegrating the cloud-edge infrastructure.

Reproducible Recipes for Bringing VLA Systems into Practice

Unified infrastructure solves the problem of building workflows, but developers still need to know how to choose configurations and reproduce results for a specific model, environment, and post-training objective. verl-vla packages models, environments, data, resource topologies, training procedures, and evaluation protocols into end-to-end recipes. Developers can run the reference configurations directly or swap in their own models, environments, or algorithms within the same workflow.

The table below lists representative recipes validated so far. They cover web-based teleoperation and demonstration collection, supervised fine-tuning, and reinforcement learning post-training with methods such as RECAP and DSRL.

These recipes provide reusable starting points for VLAs of different scales, environments, and post-training paradigms, helping developers close the loop from environment integration to policy validation more quickly.

Announcing verl-vla v0.1.0

verl-vla v0.1.0 is now available. As a unified VLA post-training framework built on verl, it lets models, environments, and training algorithms integrate independently and be composed as needed across the full post-training lifecycle: from human demonstrations and supervised fine-tuning through evaluation, reinforcement learning, and human intervention.

The current release supports policy models including ACT, Gaussian Actor, Pi0.5, and Isaac GR00T; covers the LIBERO and Isaac Lab-Arena environments; and provides post-training workflows for SFT, SAC-style off-policy training, DSRL, and RECAP. Developers can also teleoperate and intervene through keyboards, gamepads, or XR controllers while observing environment state and execution in real time through the browser.

Next, we will continue expanding the supported models, environments, input devices, post-training methods, and end-to-end recipes. Bringing physical-robot training into the unified cloud-edge post-training loop is an important direction: enabling closer coordination between real robots, cloud inference, data collection, human intervention, and continuous optimization. Visit the verl-vla GitHub repository and documentation to explore the current capabilities and contribute.